Only 18¢ of Your AI Dollar Reaches the Product. Here's How to Fix It.
Rework has always been most of the work. What changed is that generation outran review, and the two things everyone is buying more of are making it worse.
Paper 1 in the Reliable AI Systems series. Companion to Defense in Depth for AI Agents.
Written as the companion to my lightning talk at ClawCamp SF, Saturday 15 August 2026. If you just scanned the code from the room: this is the whole argument, with the sources.
Slides from the talk: https://xteamos.exponentialos.io/deck/
A survey of 2,444 companies put a number on something most engineering teams already feel.
For every dollar spent on AI tokens: 44 cents goes to fixing bugs the AI created. 27 cents to rewriting its code. 11 cents to review and merge delays.
Eighteen cents reaches production.
That number went around as “82% of your AI spend is wasted,” and I want to deal with the objection before I use it, because the objection is good.
The number is directionally right and numerically dubious
The study comes from Entelligence, a company that sells tooling for the exact problem the statistic describes. The methodology is self-reported. The 2,444 companies likely include free-tier accounts. A forensic teardown of the claim landed on the right verdict: directionally right, numerically dubious.
The teardown’s sharpest point is the one nobody quoting the stat mentions: pre-AI software development already spent 75–80% of effort on maintenance and debugging. So 82% isn’t a shocking new tax. It’s roughly the historical baseline wearing a new hat.
Which means the interesting question isn’t “how much is wasted.” It’s what actually changed.
What changed: generation outran review
Rework has always been most of the work. What’s new is the ratio between how fast you can produce code and how fast anyone can check it.
CodeRabbit measures AI-generated changes carrying roughly 1.7× more issues than human-written ones. Lightrun’s 2026 report found 43% of AI-generated code still requires manual debugging in production — after it passed quality checks.
Sit with that second one. Not “before review caught it.” After.
So the bottleneck was never cost per token. It’s review throughput, and every model upgrade makes the imbalance worse, because generation scales and human review does not.
That reframes what a quality gate is. A gate isn’t a tax on velocity. A gate is the only thing that raises the share of your spend that reaches the product. Yield, not savings.
Two things are quietly eating that yield, and the second one is the one nobody says out loud.
Leak one: you route every task as if it were the hardest one
You get access to a frontier model. It’s good at everything, so you send it everything — file reads, git operations, deploys, browser automation, boilerplate tests, and the two decisions a day that actually need judgment.
Those are not the same task. You are paying judgment prices for clerical work, and you hit the rate ceiling by mid-afternoon because you spent the budget on git status.
The fix is routing by task class rather than by habit:
1) Deterministic work — deploys, git, file operations — goes to a small fast model. On current public pricing that’s roughly 20× cheaper than the frontier tier.
2) Browser automation and large reads go to a cheap high-context model, roughly 50× cheaper. This one matters more than it sounds: a single page snapshot can dump an entire DOM into your context, and you pay for those tokens on every subsequent turn.
3) Code generation goes to a code-specialised model.
4) The expensive model does one thing: judgment. Specs, gates, synthesis. That’s it.
None of that is controversial. It’s the second leak that people argue with.
Leak two: a bigger context window makes your agent worse
Everyone has been told to start a new session when things go sideways. Almost nobody explains why.
It’s a cure prescribed without a diagnosis, and once you have the diagnosis, the advice turns out to be the crudest possible intervention.
Why a fresh session feels smarter
Not because it’s emptier. Because the wrong turns are gone.
Every dead end you explored is still sitting in that window. The three incorrect things you told it about your schema two hours ago are still there, still being conditioned on. A fresh session doesn’t give the model more room to think — it removes the accumulated wrong answers.
That distinction kills the naive fix. Compressing by volume doesn’t help if you compress the wrong turns along with the right ones. You end up with a tidy summary of your own mistakes.
The research says it’s worse than dilution
Two findings, both well-established, both counterintuitive.
Position matters, badly. Liu et al., Lost in the Middle: How Language Models Use Long Contexts (Stanford, published in TACL), found performance is highest when relevant information sits at the beginning or end of the context and degrades significantly when the model must retrieve from the middle. Put the important thing in the middle of a long window and you have hidden it.
Irrelevant context isn’t neutral — it competes. Chroma’s context-rot work and the GSM-DC benchmark both find that models are meaningfully degraded by distractors, and that semantic similarity drives decay more than length does. A distractor that looks like the answer costs you far more than one that obviously doesn’t. A single distractor already degrades baseline; several compound it.
And then the finding that should change how you think about this entirely:
Models performed better on shuffled context than on logically coherent context — across 18 models. Coherent documents share terminology and structure, which makes them better distractors.
Read that again, because it inverts the instinct everyone has. The more relevant-looking the material you stuff into the window, the more damage it does. “Just give it all the related files, it’s all connected” is precisely the wrong move.
“It fits” is not the same as “it helps.”
The architecture: the smallest thing that does the job
Both leaks have the same shape, and the same fix.
Sharp focus. Keep the working context deliberately small. Not as a limitation you’re working around — as a design choice.
Long-term memory. Everything else lives in an index outside the window. The agent doesn’t need to hold it. It needs to know where it is.
Just-in-time injection. Hydrate only the relevant slice, at the moment the task needs it. When the session ends, distil what was learned back into the index.
That’s what I run daily. The agent doesn’t carry everything. It carries the right thing at the right moment and knows where the rest lives.
Which makes “start a new session” look like what it is: amputation. It works. It also throws away everything you learned getting there. That’s only a rational trade if you have no memory layer — which, for most people giving the advice, is true.
Two better techniques:
Self-compress on your own schedule. The critical word is your. Auto-compaction fires when the system hits a limit, which is precisely the moment you have least control over what survives. Compress deliberately, at a natural boundary, and you choose what carries forward — by relevance, not by volume.
Write the handoff as insurance. You don’t get to pick when a session dies. This is the case you can’t schedule, which is exactly why it’s load-bearing.
Both convert a reset from a loss into a checkpoint.
The gates, and why they’re the yield story
Cheap routing and a small context window let you generate faster. That’s only an improvement if what you generate is right. Otherwise you’ve optimised your way into producing defects more efficiently.
So: gates. Acceptance criteria and evals fixed before code is written. Ships only if it beats baseline.
If you take one gate from this piece, take this one.
Never let the model that wrote the code be the model that reviews it.
Not a different prompt. Not a different persona. A different family. Claude reviewing Claude shares the training, so it shares the blind spot — and it will confidently approve its own mistake.
I didn’t want to believe that on vibes, so I tested it: 50 artifacts seeded with known flaws from a taxonomy of five failure modes, four review conditions, 240 review runs, hypothesis registered before a single run so I couldn’t move the goalposts afterward.
But the production evidence is blunter than the experiment.
A cross-family review panel caught a data-integrity bug and a silent cost-tracking regression that the CI gates — including 100% SonarCloud coverage — had waved through. Perfect coverage. Static analysis green. Two real defects shipped anyway, one of them quietly burning money.
That is the Lightrun 43% statistic happening in a single repository.
Two more from the same period:
A single judge passed an attribution defect that the cross-family panel flagged RED at 97% confidence. One event keyed a raw session-row id while its pair used a different identity, so conversion credit would never have joined back to the referrer. Same code, same moment — one reviewer missed it, a panel of different families caught it.
One model caught a corrupt-state bug that two models from the same other family both missed — corrupt state silently wiping configuration to defaults.
I’m not going to give you a catch rate. I don’t have a denominator, and a made-up percentage would undo everything above. What I have is a dated record of gates catching defects that coverage, unit tests, and single-model review all passed.
The tax nobody prices
Rework doesn’t only burn tokens. It burns the thing you can’t buy back.
Debugging plausible-looking code you didn’t write is a genuinely harder cognitive task than debugging your own. You have no mental model to fall back on, so you reconstruct intent from scratch. Every defect pulls someone out of build mode, and the context switch costs more than the fix.
And then the compounding one: every defect that ships through your gates teaches the developer to stop trusting the output. Once that happens, they re-read everything — and you have paid for speed and lost it.
That isn’t a soft cost. It’s the mechanism by which the entire investment decays, and it appears in none of the studies above, because nobody is measuring it.
What to change on Monday
One. Take the three most repetitive things you send to a frontier model this week and route them to a cheap one. Measure the difference. You will be annoyed at how much you were spending.
Two. Take whatever you stuff into context by default and move it behind a retrieval step. Inject it when the task needs it, not before. Then add one review pass from a different model family before anything ships.
The model isn’t your bottleneck. Your routing is — both kinds.
This is Paper 1 in the Reliable AI Systems series, which argues that AI systems must be governed at multiple layers simultaneously to be reliable in deployment. Each piece takes one layer and contributes a primitive. The companion piece, Defense in Depth for AI Agents, covers the operational layer: structured evaluation pipelines, drift monitoring, and guardrail-as-architecture.
I’m Anand Vallamsetla — Exponential OS, ex-Google, ex-AI Fund (Andrew Ng’s venture studio). I build the systems that build AI products.
Sources
Entelligence AI — token spend breakdown across 2,444 companies — https://research.entelligence.ai/
Lightrun — 2026 State of AI-Powered Engineering — 43% of AI code needs manual debugging after passing quality checks
CodeRabbit — AI-generated changes carry ~1.7× more issues than human-written
Jake Cuthbertson — The Rework Tax, forensic teardown of the 82% claim — https://jakecuth.com/work/ai-rework-lab/
Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, Liang — Lost in the Middle: How Language Models Use Long Contexts, TACL 2024 — https://arxiv.org/abs/2307.03172
How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark (GSM-DC) — https://arxiv.org/abs/2505.18761
Chroma — context rot research on distractor semantics and shuffled-vs-coherent haystacks