What this section answers
Parameter-efficient fine-tuning is cheap in compute and expensive in everything around it; RAG and fine-tuning substitute for knowledge-heavy work and complement for behavior-heavy work.
decision · section 09 of 10
If you're evaluating fine-tuning vs retrieval
When does the narrow model win, and by how much?
Boundary
Data preparation, evaluation, and MLOps dominate the total, and two named conditions widen the gap.
Source coordinate
Report heading If you're evaluating fine-tuning vs retrieval, under What this means for decisions, in The real cost of AI: August 2026.
The section itself
Three verified findings from the fine-tuning notebook, all pointing the same direction. PEFT (LoRA/QLoRA) makes the compute almost free — a 70B QLoRA run costs tens of dollars on a single H100 vs 8×H100 hours for full fine-tuning. Compute is nonetheless only 2–3% of a real programme: dataset curation, evaluation, and MLOps dominate, and two conditions widen that gap — fine-tuning causes unpredictable safety drift even on clean data (budget recurring safety evals per refit), and every base-model deprecation forces a refit, making compute a subscription rather than a purchase. On the build decision itself: RAG and fine-tuning are substitutes for knowledge-heavy tasks (RAG is updateable and cheaper to iterate) but complements for behavior-heavy tasks — fine-tune the behavior, retrieve the knowledge.
Who uses this section, and for what
Researchers and data scientists
Where the compute cost stops and the real cost starts.
Reader questions that route through here
- Is fine-tuning worth it?Answered