Platform and infrastructure teams

The sticker price of a token is not the cost of an answer. In August 2026, the same standard API workload (100K input + 20K output tokens) costs anywhere from $0.018 on Gemini Flash-Lite to $2.00 on Claude Fable 5 — a 111x price range across comparable models. Two corrections cut the other way: measured token-efficiency variance means the same input produces 2.65x+ more output tokens on some models, and cheap-token models can cost more per successful task via verbosity and retry loops — so the posted range is an upper bound on effective dispersion, and it still understates the real spread: adjusted for quality using the Artificial Analysis Intelligence Index, the cost-per-quality- point spreads by 47x. Read that 47x as order-of-magnitude evidence of non-linear pricing, not a precise ratio: the index is an editorially-weighted nine-benchmark composite (text/English-centric; agentic-weighted since v4.1), and version overhauls move scores by ~23 points — as large as the frontier spread itself. The marginal cost of capability accelerates sharply above Intelligence Index 50.

Running your own GPUs is a crossover problem, not a preference. Specialist GPU clouds (“neoclouds” like RunPod, Lambda, Together AI) charge 50-70% less than hyperscalers for the same H100: a median of $3.99/GPU-hour vs $7.89. The TCO model — now corrected for procurement scope — shows the answer splits by who’s buying: a lean operator adding GPUs to existing infrastructure (~$30K/GPU street price) breaks even against hyperscaler on-demand at ~21% utilization and beats median neocloud pricing above ~37%; an enterprise node-loaded buyer (~$94K/GPU all-in per io.net’s measured 3-year figure) breaks even vs hyperscaler on-demand only at ~57%, essentially never beats median neocloud pricing, and never beats cheap neoclouds or spot. Reserved commitments beat node-loaded on-prem at every utilization. Utilization assumption AND procurement scope together dominate the decision.

The control premium is real, and buyers pay it knowingly. Independent cost itemization puts true self-hosting at 3–5× the pure GPU price — matching the node-loaded math. For a compliance-driven subset, the premium is not optional: sovereign cloud IaaS spending is forecast at $80B in 2026 (+35.6%), Italy’s cloud market grew 20% YoY explicitly on sovereign-data demand, and the EU’s Technological Sovereignty Package (June 2026) extends data-governance obligations to AI providers. The other evidenced motives: guaranteed capacity with no rate limits or peak contention, latency for on-network workloads, vendor independence as a hedge against the list-price and subsidy moves forecast above, and very-high sustained utilization. In practice the control premium runs roughly 2–4× effective cost vs neoclouds — cost-optimal is not decision-optimal when compliance or capacity certainty is a hard constraint.

A significant fraction of current AI spend is subsidized — through provider credits, free tiers, academic programs, and below-cost pricing funded by venture capital — though the exact fraction is opaque. The effective cost of AI is materially higher than what most teams currently pay, and the subsidy distorts the build-vs-buy decision by making API inference appear cheaper than its true cost.

The GPU notebook has 7 references (a compiled 8-source evidence file grade C + direct captures including A6 grade A, A7 grade A) and 3 verified claims, plus the TCO crossover model (analysis/tco-crossover-2026-08.md):

1. C1 (verified, 3 independent sources): Specialist GPU clouds (“neoclouds” like RunPod, Lambda, Together AI) charge 50-70% less than hyperscalers for the same H100 GPU-hour: median $3.99/GPU-hr vs $7.89/GPU-hr (+98%). The ~2x hyperscaler premium is consistent across H100, A100, H200, and B200.

2. C2 (verified, 4 independent sources): On-premises GPU clusters break even with hyperscaler on-demand at roughly 50-83% sustained utilization, but rarely win against specialist GPU clouds at any utilization once fully loaded.

The GPU market is split: neoclouds at $3-4/GPU-hour, hyperscalers at $7-12. If you’re paying hyperscaler on-demand for sustained inference, you are likely overpaying by 2x. Whether owning beats renting now has a two-part answer. Procurement scope first: a lean team adding GPUs to existing infrastructure (~$30K/GPU) breaks even vs hyperscaler on-demand at ~21% utilization and beats median neocloud pricing above ~37% — but an enterprise node-loaded buyer (~$94K/GPU all-in) needs ~57% just against hyperscaler on-demand, essentially never beats median neoclouds, and never beats cheap neoclouds or spot. Utilization second: above ~70% sustained, on-prem wins on raw cost in the lean regime. And if you’re buying for sovereignty, compliance, or guaranteed capacity, you’re paying a 2–4x control premium on purpose — price it as insurance, not as infrastructure.

The forecast notebook has now asserted its dated binary bets (evidence cut 2026-08-20, resolution by August 2028):

The structural read: the 10x/year compression era is ending (fixed-quality decline is decelerating toward 1.5–5x/year and bifurcating — commodity approaching free, frontier reasoning moving up in price), so planning should treat unit cost as a shrinking but non-zero line item while total spend likely still rises (FC3). The least evidenced bets — subsidy contraction and demand elasticity — are the ones that would move budgets most.

• All prices are evidence of posted rates on 2026-08-20, not durable claims. Provider pricing changes weekly.
• The quality-adjusted cost analysis uses the Artificial Analysis Intelligence Index, which is a composite benchmark score — it does not predict performance on a specific workload. The index is also an editorially-weighted nine-benchmark composite (text/English-centric, agentic-shifted since v4.1); version overhauls have moved scores by ~23 points, so cost-per-index-point figures are order-of-magnitude evidence, not precise rankings.
• The GPU pricing data is compiled from independent sources including two grade-A direct captures (getdeploying.com, spendark.com) and a compiled evidence file; both GPU claims are verified.
• The subsidy analysis is structural (the subsidy exists, it distorts decisions) and now has a cost-side quantitative anchor: frontier-lab list pricing runs below sustainable infrastructure return (OpenAI gross margin 33% in 2025 vs its own 46% forecast, inference costs ~$8.4B and 4x YoY; Anthropic 40%) — posted API prices remain effectively investor-subsidized. The user-side subsidized fraction is still unmeasured.
• The forecast bets above are dated, binary, probability-bearing judgments over a short record — not calibrated forecasts. FC4 and FC6 have named missing baselines; their probabilities are structured priors.
• Recheck by: 2026-11-20, or after a material model release, pricing change, or hardware generation shift.