Written as the companion to my lightning talk at ClawCamp SF, Saturday 15 August 2026. If you just scanned the code from the room: this is the whole argument, with the sources.

Slides from the talk: https://xteamos.exponentialos.io/deck/

A survey of 2,444 companies put a number on something most engineering teams already feel.

For every dollar spent on AI tokens: 44 cents goes to fixing bugs the AI created. 27 cents to rewriting its code. 11 cents to review and merge delays.

Eighteen cents reaches production.

That number went around as “82% of your AI spend is wasted,” and I want to deal with the objection before I use it, because the objection is good.

The number is directionally right and numerically dubious

The study comes from Entelligence, a company that sells tooling for the exact problem the statistic describes. The methodology is self-reported. The 2,444 companies likely include free-tier accounts. A forensic teardown of the claim landed on the right verdict: directionally right, numerically dubious.

The teardown’s sharpest point is the one nobody quoting the stat mentions: pre-AI software development already spent 75–80% of effort on maintenance and debugging. So 82% isn’t a shocking new tax. It’s roughly the historical baseline wearing a new hat.

Which means the interesting question isn’t “how much is wasted.” It’s what actually changed.

What changed: generation outran review

Rework has always been most of the work. What’s new is the ratio between how fast you can produce code and how fast anyone can check it.

CodeRabbit measures AI-generated changes carrying roughly 1.7× more issues than human-written ones. Lightrun’s 2026 report found 43% of AI-generated code still requires manual debugging in production — after it passed quality checks.

Sit with that second one. Not “before review caught it.” After.

So the bottleneck was never cost per token. It’s review throughput, and every model upgrade makes the imbalance worse, because generation scales and human review does not.

That reframes what a quality gate is. A gate isn’t a tax on velocity. A gate is the only thing that raises the share of your spend that reaches the product. Yield, not savings.

Two things are quietly eating that yield, and the second one is the one nobody says out loud.

Leak one: you route every task as if it were the hardest one

You get access to a frontier model. It’s good at everything, so you send it everything — file reads, git operations, deploys, browser automation, boilerplate tests, and the two decisions a day that actually need judgment.

Those are not the same task. You are paying judgment prices for clerical work, and you hit the rate ceiling by mid-afternoon because you spent the budget on git status.

The fix is routing by task class rather than by habit:

1) Deterministic work — deploys, git, file operations — goes to a small fast model. On current public pricing that’s roughly 20× cheaper than the frontier tier.

2) Browser automation and large reads go to a cheap high-context model, roughly 50× cheaper. This one matters more than it sounds: a single page snapshot can dump an entire DOM into your context, and you pay for those tokens on every subsequent turn.

3) Code generation goes to a code-specialised model.

4) The expensive model does one thing: judgment. Specs, gates, synthesis. That’s it.

None of that is controversial. It’s the second leak that people argue with.

Leak two: a bigger context window makes your agent worse

Everyone has been told to start a new session when things go sideways. Almost nobody explains why.

It’s a cure prescribed without a diagnosis, and once you have the diagnosis, the advice turns out to be the crudest possible intervention.

Why a fresh session feels smarter

Not because it’s emptier. Because the wrong turns are gone.

Every dead end you explored is still sitting in that window. The three incorrect things you told it about your schema two hours ago are still there, still being conditioned on. A fresh session doesn’t give the model more room to think — it removes the accumulated wrong answers.

That distinction kills the naive fix. Compressing by volume doesn’t help if you compress the wrong turns along with the right ones. You end up with a tidy summary of your own mistakes.

The research says it’s worse than dilution

Two findings, both well-established, both counterintuitive.

Position matters, badly. Liu et al., Lost in the Middle: How Language Models Use Long Contexts (Stanford, published in TACL), found performance is highest when relevant information sits at the beginning or end of the context and degrades significantly when the model must retrieve from the middle. Put the important thing in the middle of a long window and you have hidden it.

Irrelevant context isn’t neutral — it competes. Chroma’s context-rot work and the GSM-DC benchmark both find that models are meaningfully degraded by distractors, and that semantic similarity drives decay more than length does. A distractor that looks like the answer costs you far more than one that obviously doesn’t. A single distractor already degrades baseline; several compound it.

And then the finding that should change how you think about this entirely:

Models performed better on shuffled context than on logically coherent context — across 18 models. Coherent documents share terminology and structure, which makes them better distractors.

Read that again, because it inverts the instinct everyone has. The more relevant-looking the material you stuff into the window, the more damage it does. “Just give it all the related files, it’s all connected” is precisely the wrong move.

“It fits” is not the same as “it helps.”

The architecture: the smallest thing that does the job

Both leaks have the same shape, and the same fix.

Sharp focus. Keep the working context deliberately small. Not as a limitation you’re working around — as a design choice.

Long-term memory. Everything else lives in an index outside the window. The agent doesn’t need to hold it. It needs to know where it is.

Just-in-time injection. Hydrate only the relevant slice, at the moment the task needs it. When the session ends, distil what was learned back into the index.

That’s what I run daily. The agent doesn’t carry everything. It carries the right thing at the right moment and knows where the rest lives.

Which makes “start a new session” look like what it is: amputation. It works. It also throws away everything you learned getting there. That’s only a rational trade if you have no memory layer — which, for most people giving the advice, is true.

Two better techniques:

Self-compress on your own schedule. The critical word is your. Auto-compaction fires when the system hits a limit, which is precisely the moment you have least control over what survives. Compress deliberately, at a natural boundary, and you choose what carries forward — by relevance, not by volume.

Write the handoff as insurance. You don’t get to pick when a session dies. This is the case you can’t schedule, which is exactly why it’s load-bearing.

Both convert a reset from a loss into a checkpoint.

The gates, and why they’re the yield story

Cheap routing and a small context window let you generate faster. That’s only an improvement if what you generate is right. Otherwise you’ve optimised your way into producing defects more efficiently.

So: gates. Acceptance criteria and evals fixed before code is written. Ships only if it beats baseline.

If you take one gate from this piece, take this one.

Never let the model that wrote the code be the model that reviews it.

Not a different prompt. Not a different persona. A different family. Claude reviewing Claude shares the training, so it shares the blind spot — and it will confidently approve its own mistake.

I didn’t want to believe that on vibes, so I tested it: 50 artifacts seeded with known flaws from a taxonomy of five failure modes, four review conditions, 240 review runs, hypothesis registered before a single run so I couldn’t move the goalposts afterward.

But the production evidence is blunter than the experiment.

A cross-family review panel caught a data-integrity bug and a silent cost-tracking regression that the CI gates — including 100% SonarCloud coverage — had waved through. Perfect coverage. Static analysis green. Two real defects shipped anyway, one of them quietly burning money.

That is the Lightrun 43% statistic happening in a single repository.

Two more from the same period:

A single judge passed an attribution defect that the cross-family panel flagged RED at 97% confidence. One event keyed a raw session-row id while its pair used a different identity, so conversion credit would never have joined back to the referrer. Same code, same moment — one reviewer missed it, a panel of different families caught it.

One model caught a corrupt-state bug that two models from the same other family both missed — corrupt state silently wiping configuration to defaults.

I’m not going to give you a catch rate. I don’t have a denominator, and a made-up percentage would undo everything above. What I have is a dated record of gates catching defects that coverage, unit tests, and single-model review all passed.

The tax nobody prices

Rework doesn’t only burn tokens. It burns the thing you can’t buy back.

Debugging plausible-looking code you didn’t write is a genuinely harder cognitive task than debugging your own. You have no mental model to fall back on, so you reconstruct intent from scratch. Every defect pulls someone out of build mode, and the context switch costs more than the fix.

And then the compounding one: every defect that ships through your gates teaches the developer to stop trusting the output. Once that happens, they re-read everything — and you have paid for speed and lost it.

That isn’t a soft cost. It’s the mechanism by which the entire investment decays, and it appears in none of the studies above, because nobody is measuring it.

What to change on Monday

One. Take the three most repetitive things you send to a frontier model this week and route them to a cheap one. Measure the difference. You will be annoyed at how much you were spending.

Two. Take whatever you stuff into context by default and move it behind a retrieval step. Inject it when the task needs it, not before. Then add one review pass from a different model family before anything ships.

The model isn’t your bottleneck. Your routing is — both kinds.

This is Paper 1 in the Reliable AI Systems series, which argues that AI systems must be governed at multiple layers simultaneously to be reliable in deployment. Each piece takes one layer and contributes a primitive. The companion piece, Defense in Depth for AI Agents, covers the operational layer: structured evaluation pipelines, drift monitoring, and guardrail-as-architecture.

I’m Anand Vallamsetla — Exponential OS, ex-Google, ex-AI Fund (Andrew Ng’s venture studio). I build the systems that build AI products.

Sources

• Entelligence AI — token spend breakdown across 2,444 companies — https://research.entelligence.ai/

• Lightrun — 2026 State of AI-Powered Engineering — 43% of AI code needs manual debugging after passing quality checks

• CodeRabbit — AI-generated changes carry ~1.7× more issues than human-written

• Jake Cuthbertson — The Rework Tax, forensic teardown of the 82% claim — https://jakecuth.com/work/ai-rework-lab/

• Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, Liang — Lost in the Middle: How Language Models Use Long Contexts, TACL 2024 — https://arxiv.org/abs/2307.03172

• How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark (GSM-DC) — https://arxiv.org/abs/2505.18761

• Chroma — context rot research on distractor semantics and shuffled-vs-coherent haystacks

That gap is the whole story. Ranking on Google and being citable by AI are not the same problem, and treating them as one is why so many sites have gone quiet in AI answers without noticing.

Here's what's actually happening under the hood.

An AI answer comes from one of two places. Either the model answers from memory — what it absorbed in training, which you can't change and which is months stale — or it runs a live retrieval, fetches pages, and cites them. Only the second path is one you can influence. And that path has two gates almost nobody checks.

Gate 1: Can the crawler even fetch you?

The retrieval bots behind ChatGPT, Claude, and Perplexity — OAI-SearchBot, ClaudeBot, PerplexityBot — do not run JavaScript. A large-scale study of roughly a billion requests found none of them render JS. So if your site builds its content in the browser (any SPA — React, Vue, most modern stacks), the crawler sees an almost-empty shell where a human sees a full page. You can rank on Google, which does render JS, and be a blank page to ChatGPT.

The other half of Gate 1 is your own security layer. A WAF or Cloudflare rule that challenges "bot-like" traffic will 403 the AI crawler while serving humans normally. You never see it. You just quietly disappear.

Here's the ten-second test. Run this on your own site:

curl -A "ClaudeBot/1.0 (+https://www.anthropic.com/claude-bot)" https://yoursite.com

If you get your real content back, good. If you get an empty shell, a redirect loop, or a 403 — that's why you're not cited, and no amount of content strategy fixes it until you fix this.

Gate 2: Once fetched, the index decides — and every engine's index is different

I ran one experiment that made this vivid. I asked four assistants the same open question — "best API to extract structured data from PDFs" — each on its own search index.

• ChatGPT (Bing index): surfaced exact-match-domain microsites — frompdf.dev, transpdf.ai, extractocr.com. When I actually fetched frompdf.dev, it returned zero bytes — no readable content at all — yet it was cited the most. These sites have no measurable traffic and zero presence on Reddit. They won on one thing: their domain name is the query.

• Gemini (Google index): Google Document AI, AWS Textract, Azure, LlamaParse, Unstructured — plus LandingAI, a legitimate, well-built player with a fraction of the incumbents' traffic.

• Claude (Brave index): cloud incumbents and real tools; Parseur, Klippa, LlamaParse.

Same question. Four different worlds. On Bing/ChatGPT, keyword-stuffed domains with no audience beat a readable, funded product. On Google and Brave, legitimacy won. "Getting cited by AI" isn't one target — it's four, and they reward different things.

There's a concentration story on top of this. One synthesis of 680M+ AI citations (Everything-PR / 5WPR, 2026) found the top 15 domains capture ~68% of all citations, with Reddit alone near 40% and #1 on every engine. (It's a third-party aggregation, not an audited study — treat it as directional.) Much of AI citation flows through a handful of high-trust intermediaries you don't own — a different fight from making your own site citable.

What doesn't work, despite the hype

llms.txt. One audit of 1,500 sites found 0.2% use it, and real crawler consumption today is near zero. Schema/JSON-LD helps a parser understand you but doesn't get you retrieved. Both are fine hygiene; neither is the lever people think it is.

So, in order of impact

1. Make sure AI crawlers can fetch you — server-render or pre-generate your main content, and stop challenging verified bots at the edge. (The curl test above.)

2. Put concrete, quotable facts in the first few hundred words, not boilerplate.

3. Optimize per engine, not in general — the index that feeds your buyers' assistant is the one that matters.

4. llms.txt last, if at all.

None of this requires a rebrand. It requires knowing what an AI agent actually sees when it reads you — which, it turns out, is almost never what you see in a browser.

(I got tired of running that curl test by hand, so I built a free scanner that fetches your page as ChatGPT and Claude's crawlers do and shows the exact gaps — no signup: getagentview.com. But the curl one-liner above will get you 80% of the way for free.)

Caveats, because they matter: the four-engine test is a single snapshot, not a controlled study — results drift week to week. The citation-share figures are third-party aggregations. And correlation isn't causation — reachability tracks closely with citation in my testing, but the clean proof is a before/after on a site you actually fix.