The Inference Cost Paradox: Token Prices Fall, AI Spend Explodes

AI Overview

The price of AI inference — the cost to run a model and get an answer — is falling fast. By one widely cited estimate from Andreessen Horowitz, the cost of a fixed level of language-model capability dropped about 10x per year between 2021 and 2024: roughly $60 per million tokens down to $0.06. Yet total spending on inference, across the industry and inside most individual products, keeps rising. Both are true at the same time, and the reason is not a contradiction. It is the Jevons paradox: when a resource becomes cheaper to use, cheaper use unlocks so much new demand that total consumption grows rather than shrinks. Falling token prices do not hand most builders a smaller bill. They hand them a larger market — every price cut makes another tier of use cases economical, so volume climbs faster than unit cost falls. The strategic mistake is to budget as if cheaper tokens mean you spend less. The right question is how much new usage each price cut unlocks in your product — an elasticity question, not a pricing one.

In Short

• Per-token AI prices are falling roughly 10x per year for equivalent capability; total inference spend is rising anyway.
• The mechanism is the Jevons paradox: efficiency lowers the price of use, and lower prices expand demand faster than they cut the bill.
• For most builders, a price cut is a demand accelerant, not margin relief — your volume grows faster than your unit cost drops.
• Whether you gain margin or just more traffic depends on the elasticity of your own demand, which is specific to your workload.
• The Inference Spend Decomposition in this piece separates the three forces so you can tell which side of the paradox you are on.

Key Facts

Fact	Value
Category	Artificial Intelligence
Difficulty	Intermediate
Reading time	11 min
Search intent	Informational
Updated	July 2026

The AI cost conversation is stuck on a single number: the price of a token. That number is falling so reliably that it has its own nickname — a16z called it "LLMflation" — and the natural conclusion is that AI is getting cheaper. For a fixed task, it is. But "the price of a token is falling" and "AI is getting cheaper for me" are different claims, and conflating them is why finance teams keep getting surprised by the bill.

The gap between them is the whole story. A falling unit price does not settle what you spend; it only sets one of the two terms in the product spend = volume × price. When a price cut moves volume more than it moves price, spend rises. That is not a market failure or a pricing trick. It is the oldest result in resource economics, and it applies to tokens as cleanly as it applied to coal.

This article does three things: it establishes both facts with public data, explains the mechanism that makes them compatible, and gives you a decomposition you can apply to your own product to predict which way your bill moves when prices fall again.

Why It Matters

For builders, this reframes the most common budgeting error in AI products. Teams see published prices drop and forecast a shrinking cost line. Then usage — theirs and their customers' — expands into the space the price cut opened, and the line goes the other way. The error is not bad math; it is modeling price while ignoring the demand it unleashes.

Economically, it explains why cheaper inference has coincided with record capital spending, not restraint. Microsoft, Alphabet, Amazon and Meta have all guided capital expenditure sharply upward through this period even as per-token prices collapsed — Microsoft alone reported tens of billions in quarterly capital spending in its FY2025 earnings. Falling prices are not evidence the boom is cooling. Under the paradox, they are part of what drives it.

Strategically, it changes what counts as a moat. If the price of raw capability falls toward zero, access to a cheap model is not an advantage — everyone gets it. The advantage is owning a workload whose value per token rises, so you capture more of each token's output even as each token costs less.

Core Concepts

Inference is the cost of running a trained model to produce an output, as opposed to training, the one-time cost of building it. Inference is a recurring, per-use cost, which is why its unit price drives product economics. A token is the unit models read and write in — roughly three-quarters of a word — and the industry prices inference per million tokens. The Jevons paradox, named for the economist William Stanley Jevons, is the observation that improving the efficiency with which a resource is used tends to increase total consumption of that resource, because efficiency lowers the effective price and lower prices expand demand (overview). Demand elasticity is how much the quantity consumed changes when the price changes; it is the hinge on which the whole argument turns.

The First Fact: Prices Are Falling, Fast

The decline in inference cost is real, large, and well-documented. a16z's "LLMflation" analysis put it plainly: for a language model of equivalent performance, inference cost fell roughly 10x every year — what cost about $60 per million tokens in 2021 cost about $0.06 by late 2024 (a16z, 2024). That is close to a 1,000x decline over three years for a fixed capability level.

Three forces compound to produce it:

Force	What changes	Effect on cost per token
Hardware	More FLOPs per dollar, better memory bandwidth	Falls with each accelerator generation
Model efficiency	Smaller models matching older large ones; quantization; distillation	A capability that needed a frontier model now runs on a fraction of the compute
Serving optimization	Batching, caching, speculative decoding	More useful output per GPU-second

None of these is speculative. Speculative decoding — running a small model to draft tokens a large model then verifies — is a published, deployed technique for cutting latency and cost without changing outputs (Leviathan et al., 2022). The efficiency frontier kept moving in 2025: DeepSeek-V3 demonstrated frontier-class performance trained and served at a fraction of the assumed cost, which is exactly the kind of event that resets the price floor downward for everyone. Underlying all of it, the scaling laws that predicted capability from compute also imply that efficiency gains translate directly into cost relief for a fixed target.

The direction is not in dispute. The Stanford AI Index tracks the same decline across its cost datasets, as does Epoch AI. For any fixed task, AI inference is getting cheaper on a curve steep enough to be almost unique in the history of computing.

The Second Fact: Total Spend Is Rising Anyway

Now the part that should be surprising and is not. Over the same window that per-token prices fell roughly 1,000x, total spending on AI infrastructure went up — steeply, and at the level of the largest buyers. The hyperscalers guided capital expenditure higher quarter after quarter through 2024 and 2025, and the investment case that frames the whole build-out — Sequoia's "AI's $600B question" — is about revenue needing to catch up to spend, not spend coming down.

The same pattern holds inside individual products, one layer down. A team that ran a model on 10 million tokens a month at $60 per million — a $600 bill — does not, when the price drops to $0.06, keep sending 10 million tokens and pocket a $0.60 bill. It finds that at $0.06 per million, a hundred use cases that were uneconomical at $60 are suddenly worth running. Usage climbs into the space the price opened. The bill grows.

That is the paradox stated in one product's ledger: the unit got 1,000x cheaper and the customer spent more. Multiply that across every team discovering the same thing at the same time, and industry-wide spend rises even as every published price falls.

The Synthetic Break From Coal

It is worth naming where the analogy strains, because the mechanism only holds if the demand is really there. Jevons's coal had effectively unlimited latent demand — an entire industrializing economy waiting for cheaper energy. Whether AI inference has the same depth of latent demand is the open question. If it does, the paradox runs for years. If the truly valuable use cases are a smaller set than the hype implies, then at some price point demand saturates, and falling prices finally do produce falling bills. The paradox is a description of the current regime, not a physical law that guarantees it continues. Which brings us to the framework.

The Inference Spend Decomposition

Everything above collapses into one relationship you can compute for your own product. Total inference spend over a period is:

Spend = Volume × Price

and the change in spend when price falls depends entirely on how much volume responds. Decompose it into three forces and you can predict your own bill:

1. The Price Force (down). The market price per token, falling on the LLMflation curve. You do not control it; you inherit it. Treat its decline as a given, not a saving.

2. The Elasticity Force (up, and the one that decides everything). How much new volume each price cut unlocks in your product. High elasticity — a workload where cheaper tokens open many new use cases (agents that call models thousands of times, background enrichment, "run it on everything" features) — means the price cut is spent on volume and your bill rises. Low elasticity — a fixed, bounded workload (one summary per document, a set number of support tickets) — means the price cut actually reaches the bottom line.

3. The Value Force (the moat). How fast the value you extract per token rises. This is the only force you fully own. A workload where each token is worth more over time — because the output feeds a higher-value decision, or the model does more per call — lets you stay profitable no matter what the token costs.

The decomposition's blunt conclusion: cheaper tokens help you only where your demand is inelastic or your value-per-token is rising. Everywhere else, a price cut is not margin — it is an invitation to consume more. Most AI products are high-elasticity by design (that is what "AI-native" usually means), which is precisely why the industry's bill rises as its prices fall.

The practical move is to locate your workload on the elasticity axis honestly, then either accept that you are a volume business and price accordingly, or move up the value axis until each token earns more than it costs — the same discipline that separates durable AI products from ones that fail at 10x usage.

Common Misconceptions

Myth: Falling inference prices mean AI is getting cheaper to operate. Reality: A fixed task is getting cheaper. Your product gets cheaper only if its usage does not expand into the price cut — and most AI products are built specifically to expand usage.

Myth: The capex boom must slow now that inference is cheap. Reality: Cheaper inference expands the addressable set of workloads, which is part of what justifies more infrastructure spend. Falling prices and rising capex are the two halves of the same paradox, not a contradiction.

Myth: The cheapest model provider wins. Reality: When raw capability commoditizes, price is table stakes. The durable advantage is a workload whose value per token rises — see the economics of a real moat.

Limitations

The paradox describes a regime; it is not guaranteed to last. First, the 10x-per-year decline will slow — the fastest gains come early in any technology, and specific crossover points computed today have a short shelf life. Second, the mechanism depends on latent demand actually existing; if high-value use cases are scarcer than assumed, demand saturates and falling prices eventually do cut bills, breaking the coal analogy (as noted above). Third, elasticity is empirical and product- specific — you estimate it by watching how your own usage responds to price and capability changes, not from anyone's benchmark. Fourth, this analysis holds at mid-2026 pricing and efficiency; the direction is durable but any specific number here is a snapshot. Track the annual cost and capability data when refreshing the figures (Stanford AI Index).

FAQ

The questions below reflect what builders and finance teams actually ask when the bill and the price sheet disagree.

Explore Related Concepts

AI Compute · Inference Optimization · Economic Moats · Large Language Models

Related Analysis

• Artificial Intelligence hub — the parent topic hub
• Building AI Systems That Actually Work — the AI-systems pillar this sits under
• The Inference Economics Crisis — the companion argument: where per-token cost has plateaued and products run underwater
• The Thinking Premium: What Reasoning AI Actually Costs — how reasoning models change the per-call cost math
• Scaling Is Changing Shape — the research redrawing the compute-and-cost map

References

1. Andreessen Horowitz. "Welcome to LLMflation — LLM inference cost is going down fast." (2024). https://a16z.com/llmflation-llm-inference-cost/
2. Leviathan, Y., Kalman, M., Matias, Y. "Fast Inference from Transformers via Speculative Decoding." arXiv (2022). https://arxiv.org/abs/2211.17192
3. DeepSeek-AI. "DeepSeek-V3 Technical Report." arXiv (2024). https://arxiv.org/abs/2412.19437
4. Kaplan, J. et al. "Scaling Laws for Neural Language Models." arXiv (2020). https://arxiv.org/abs/2001.08361
5. Stanford HAI. AI Index Report. https://hai.stanford.edu/ai-index
6. Epoch AI. "Large-scale AI models" dataset. https://epoch.ai/data/large-scale-ai-models
7. Sequoia Capital. "AI's $600B Question." https://www.sequoiacap.com/article/ais-600b-question/
8. Microsoft. FY2025 Q3 earnings release. https://www.microsoft.com/en-us/investor/earnings/fy-2025-q3/press-release-webcast
9. "Jevons paradox." https://en.wikipedia.org/wiki/Jevons_paradox

Final Thoughts

The inference-cost paradox is not a puzzle once you stop treating a falling price as a falling bill. Price is one term; demand is the other, and in a technology built to find new uses, demand is the term that moves. The teams that will be surprised by their AI spend are the ones still forecasting from the price sheet. The teams that won't are the ones who have measured their own elasticity and moved their workloads up the value axis — where each token, however cheap, earns more than it costs. Cheaper tokens are not the end of the cost problem. They are the start of a different one: what is worth doing now that so much more is affordable?