The Economics of Open-Weight Inference

Open-weight AI models running on self-hosted older GPUs like the A100 can be significantly cheaper than closed commercial models, with costs as low as $0.12-$0.35 per million tokens compared to roughly five times higher costs for comparable closed models. This cost advantage extends the economic lifespan of older NVIDIA GPU generations, with A100s retaining 80% of their rental value over five-year contracts, challenging the assumption that newer hardware generations render previous ones obsolete. The finding reflects growing market demand for compute-intensive, latency-tolerant workloads like batch processing and reinforcement learning that prioritize cost efficiency over hardware recency.