Procurement and finance teams

The sticker price of a token is not the cost of an answer. In August 2026, the same standard API workload (100K input + 20K output tokens) costs anywhere from $0.018 on Gemini Flash-Lite to $2.00 on Claude Fable 5 — a 111x price range across comparable models. Two corrections cut the other way: measured token-efficiency variance means the same input produces 2.65x+ more output tokens on some models, and cheap-token models can cost more per successful task via verbosity and retry loops — so the posted range is an upper bound on effective dispersion, and it still understates the real spread: adjusted for quality using the Artificial Analysis Intelligence Index, the cost-per-quality- point spreads by 47x. Read that 47x as order-of-magnitude evidence of non-linear pricing, not a precise ratio: the index is an editorially-weighted nine-benchmark composite (text/English-centric; agentic-weighted since v4.1), and version overhauls move scores by ~23 points — as large as the frontier spread itself. The marginal cost of capability accelerates sharply above Intelligence Index 50.

Running your own GPUs is a crossover problem, not a preference. Specialist GPU clouds (“neoclouds” like RunPod, Lambda, Together AI) charge 50-70% less than hyperscalers for the same H100: a median of $3.99/GPU-hour vs $7.89. The TCO model — now corrected for procurement scope — shows the answer splits by who’s buying: a lean operator adding GPUs to existing infrastructure (~$30K/GPU street price) breaks even against hyperscaler on-demand at ~21% utilization and beats median neocloud pricing above ~37%; an enterprise node-loaded buyer (~$94K/GPU all-in per io.net’s measured 3-year figure) breaks even vs hyperscaler on-demand only at ~57%, essentially never beats median neocloud pricing, and never beats cheap neoclouds or spot. Reserved commitments beat node-loaded on-prem at every utilization. Utilization assumption AND procurement scope together dominate the decision.

The control premium is real, and buyers pay it knowingly. Independent cost itemization puts true self-hosting at 3–5× the pure GPU price — matching the node-loaded math. For a compliance-driven subset, the premium is not optional: sovereign cloud IaaS spending is forecast at $80B in 2026 (+35.6%), Italy’s cloud market grew 20% YoY explicitly on sovereign-data demand, and the EU’s Technological Sovereignty Package (June 2026) extends data-governance obligations to AI providers. The other evidenced motives: guaranteed capacity with no rate limits or peak contention, latency for on-network workloads, vendor independence as a hedge against the list-price and subsidy moves forecast above, and very-high sustained utilization. In practice the control premium runs roughly 2–4× effective cost vs neoclouds — cost-optimal is not decision-optimal when compliance or capacity certainty is a hard constraint.

A significant fraction of current AI spend is subsidized — through provider credits, free tiers, academic programs, and below-cost pricing funded by venture capital — though the exact fraction is opaque. The effective cost of AI is materially higher than what most teams currently pay, and the subsidy distorts the build-vs-buy decision by making API inference appear cheaper than its true cost.

Decision-makers shouldn’t need an ML background to use this evidence. Plain-language definitions of the terms doing the heaviest work:

• Token — the unit of text models read and write (~¾ of a word). All pricing starts here, which is why token counts, not just rates, matter.
• PEFT / LoRA / QLoRA — techniques that adapt a large model by training a tiny fraction of it. Makes fine-tuning compute cheap enough to run on one GPU — but doesn’t shrink the surrounding programme costs (data, evaluation, refits).
• Neocloud — a specialist GPU cloud (RunPod, Lambda, Together AI) renting raw GPU-hours at roughly half of hyperscaler prices, typically without the enterprise contracting layer.
• On-demand / reserved / spot — three rental tiers: pay-per-hour at list; 30–62% off for committing 1–3 years; and 60–91% off with interruption risk (the machine can be taken back mid-job).
• Prompt caching — paying once to process a repeated context (system prompts, documents), then 50–90% less to re-read it. Cache writes cost 1.25–2× base, so it only pays when context repeats.
• Batch API — half price on all tokens in exchange for results within 24 hours instead of seconds.
• Blended price — input and output rates combined into one number using a fixed ratio, so different models can be compared on one axis.
• Cost per successful task — what this report argues you should actually measure: total tokens (including retries and verbosity) divided by tasks completed acceptably. The only unit that reflects quality differences.
• Artificial Analysis Intelligence Index — an independent composite score (nine benchmarks, editorially weighted, text/English-centric) used here as the quality axis. Useful for locating the non-linear region; not precise enough to rank models to a decimal.
• Jevons paradox — the pattern where cheaper units increase total consumption enough that total spend rises. Bet FC3 asserts it applies here.
• Control premium — the 2–4× effective cost sovereign/compliance buyers knowingly pay for owning infrastructure rather than renting it.
• Procurement scope — whether “owning” means buying GPUs into existing infrastructure (~$30K/GPU) or procuring complete nodes with networking and financing (~$94K/GPU). The single assumption that swings every ownership crossover band.
• Utilization — the share of an owned GPU’s time doing paid work. The load-bearing variable in every own-vs-rent comparison.
• :free variants — aggregator-hosted model endpoints priced at literally $0/token with request caps. Real usage flows through them; they are also the first thing subsidy contraction would remove.

Instrument cost per successful task, not cost per token. The 111x price range across models means a routing decision (cheap model for easy queries, expensive model for hard ones) is the single largest cost lever available — but the routing infrastructure itself has a cost that must be accounted for, and the savings are quality-conditioned: a measured 8-week pilot realized 58% cost reduction at a 91% response-acceptance rate, so the acceptance threshold you set — and the residual quality cost it implies — lands on you, not the router.

The forecast notebook has now asserted its dated binary bets (evidence cut 2026-08-20, resolution by August 2028):

The structural read: the 10x/year compression era is ending (fixed-quality decline is decelerating toward 1.5–5x/year and bifurcating — commodity approaching free, frontier reasoning moving up in price), so planning should treat unit cost as a shrinking but non-zero line item while total spend likely still rises (FC3). The least evidenced bets — subsidy contraction and demand elasticity — are the ones that would move budgets most.

What’s durable vs what’s a bet

For the shortest defensible account: three findings are durable on current evidence — the posted-price range is enormous and effective dispersion narrows only under task-level accounting (C1/C2, five independent sources each); owning GPUs is a procurement-scope decision with quantified crossover bands, not a preference; and the routing layer is thin-fee and commoditizing, with consolidation already begun. Three positions are bets, not facts — that unit- cost decline continues at 1.5–5x/year (FC1), that demand absorbs the savings (FC3), and that subsidies contract (FC4). The discipline for a reader: build on the durable findings, monitor the bets by their named triggers, and treat any plan that requires all six bets to resolve favorably as a plan with no margin.

• All prices are evidence of posted rates on 2026-08-20, not durable claims. Provider pricing changes weekly.
• The quality-adjusted cost analysis uses the Artificial Analysis Intelligence Index, which is a composite benchmark score — it does not predict performance on a specific workload. The index is also an editorially-weighted nine-benchmark composite (text/English-centric, agentic-shifted since v4.1); version overhauls have moved scores by ~23 points, so cost-per-index-point figures are order-of-magnitude evidence, not precise rankings.
• The GPU pricing data is compiled from independent sources including two grade-A direct captures (getdeploying.com, spendark.com) and a compiled evidence file; both GPU claims are verified.
• The subsidy analysis is structural (the subsidy exists, it distorts decisions) and now has a cost-side quantitative anchor: frontier-lab list pricing runs below sustainable infrastructure return (OpenAI gross margin 33% in 2025 vs its own 46% forecast, inference costs ~$8.4B and 4x YoY; Anthropic 40%) — posted API prices remain effectively investor-subsidized. The user-side subsidized fraction is still unmeasured.
• The forecast bets above are dated, binary, probability-bearing judgments over a short record — not calibrated forecasts. FC4 and FC6 have named missing baselines; their probabilities are structured priors.
• Recheck by: 2026-11-20, or after a material model release, pricing change, or hardware generation shift.