A follow-up to the 18¢ piece. That one measured the cost bar. This one measures the latency bar. The goal — precision — never moves.

I was at a dinner for AI leaders at a large bank this week. I asked one of the engineering VPs the question I always ask: what is actually in the way? Not the roadmap — what stops the thing from shipping.

She answered immediately, without having to think about it. Cost against latency.

I asked what models they run. She declined, which is the right answer to that question from someone she met an hour earlier, and I did not press. But the constraint itself she stated plainly, as the thing her organisation is up against.

That stayed with me, because it is not the argument I made in the 18¢ piece. That one was about where the money goes and had nothing to say about time. I had a gap between what I had published and what someone actually running one of these platforms told me was hard.

So I measured it. Not next quarter — the next day, on the only system where I control every variable.

What I am actually measuring

There is an AI concierge on thewhyman.com. A recruiter or hiring manager lands on the page and asks it questions about me — how large a team have you run, what is your reliability record, where has your judgment failed — and it answers from a knowledge base I maintain.

It is a small system with a serious job, and that combination is why it is a useful proxy:

• Wrong is expensive. If it drops a number or invents one, that is my credibility with someone deciding whether to interview me. There is no "eh, close enough."

• Slow is visible. Nobody waits ten seconds on a personal site. They leave, and I never learn they were there.

• The answers are numbers-heavy — team sizes, portfolio scale, availability figures — which makes precision measurable instead of a matter of taste.

The stack, so you can map it onto yours:

• Serving — Cloudflare Workers AI, streaming responses at the edge.

• Shape — retrieval into a prompt. The same shape as most enterprise RAG.

• Knowledge base — about 15,000 tokens total, of which a typical answer needs a fraction.

• Retrieval — deterministic keyword selection over labelled blocks, not vector search, so I can tell exactly what the model saw for any given answer.

• Volume — low. One site. ⚠️ Nothing here is a scale claim.

⭐ What transfers is not my numbers — it is the method, and the size of the error I found by re-running it. A bank at volume has different absolutes and the same structure: retrieval into a prompt, a quality bar that cannot move, a cost line somebody reviews monthly, and a latency budget a human is sitting inside.

Six models, same knowledge base, same question, five runs each.

It was never a triangle

Cost, latency and precision get drawn as a triangle. Three things in tension, pick two. That framing is in most vendor decks and it is wrong in a way that costs you the decision.

They are not three of a kind. Precision is the goal. Cost and latency are bars.

A bar is a threshold. You clear it or you do not, and clearing it by more buys you nothing — nobody thanks you for 200ms when the SLA was two seconds. Precision does not work that way. It is the thing you are maximising, and there is no point at which more of it stops being worth having.

Everything else is a dial.

That line replaces the triangle, and it tells you what to do when a model disappoints you. In the results below there is a model that cleared both bars comfortably and still lost, and a model that hit the goal and blew a bar. Neither is a tradeoff. One violated a constraint. The other missed the point. You fix those differently, and a triangle cannot tell you which you are looking at.

What the numbers said

Median of five runs per model, after a warm-up call I throw away. Failed responses counted separately, never averaged in.

⭐⭐ The model I had in production was the worst of the four that worked, on all three axes. Most expensive. Slowest. No more accurate. Something in that table is 80% cheaper, 1.4 seconds faster, and drops exactly as many figures.

Nothing about that is a tradeoff. It is an unmeasured default, which is a different and more embarrassing thing.

And the reasoning model did not lose on latency — it never finished. Five of five runs failed against a forty-five-second ceiling. For this workload it cannot clear the bar at any price. It was buying deliberation on a task with almost nothing to deliberate about: pull the right figures out of supplied context and say them plainly.

Three method notes, because each exists where the naive version lied to me:

• Throw away the first call. A cold start recorded 137 seconds and made one model look like the cheapest winner at two samples. It was failing two runs in three.

• Medians, not means. Cost varied 0.7% across five runs; precision swung between 3/4 and 4/4 with nothing changed. Cost is stable enough to publish from few samples. Accuracy is not.

• Count failures separately. A request returning nothing in four tenths of a second is the fastest number in the set. Worse, I was still billed for it. Failure is not the absence of cost. It is cost with no product.

📌 Which points at the thing underneath: I did not benchmark six models. I benchmarked one platform's serving of six models, on one afternoon. That is the honest reason to run your own numbers — public benchmarks answer a different question than the one you have.

The drift nobody named

Here is the part I did not expect, and the one I would put in front of an executive.

Model drift is named, watched and tooled. Everyone knows the shape: the world moves, inputs shift, answers get worse. There are dashboards, alerts, vendors.

Look at what that does to your safeguards. Quality evals pass — output unchanged. SLOs green — nothing got slower. Users content — they cannot tell. Every instrument reports healthy while your cost per unit of work climbs for a quarter.

⭐ And notice who owns each. Model drift is an engineering alarm with a team that knows what to do. Cost drift arrives as a line item that grew without explanation, in a room where nobody has an engineering dashboard. The more senior problem, with the less mature tooling.

I want to be precise about my own case, because the obvious version of this story is not what happened. I did measure when I chose the model. I ran the comparison, picked the winner on evidence, shipped it. The embarrassing part is not that I skipped the work — it is that the answer expired and I did not notice. Two of the models that beat it this week did not exist when I chose.

That is a new failure mode, not dependency drift renamed. In conventional software your dependency graph changes only when you change it, and pinning is a complete defence. Here it is not — because pinning protects correctness, and what decayed was optimality. My pinned model kept working: same accuracy, no errors, nothing in any log. It quietly became the most expensive way to get that result, because correct is absolute and best is relative to a menu somebody else controls.

There is no lockfile for that. No test fails.

Continuous Evaluation

Years ago I gave a talk at SXSW about continuous integration, delivery and innovation — that shipping is not an event you schedule but a capability you maintain. This is the same argument wearing different clothes.

So a one-time bake-off is the wrong shape. It answers which model is best today, and the honest shelf life on that is somewhere between a quarter and a month.

⭐⭐ If you already run continuous evals — and most serious teams do — this is not new infrastructure. Your suite runs on a schedule and gates releases. It almost certainly measures quality alone. Put cost and latency in the harness you already own. Same fixtures, same schedule, same gate. Three columns instead of one.

That small change makes two things visible that were structurally invisible:

• Regressions you ship blind. A prompt change that adds context is a cost increase, and nothing in a quality-only suite flags it. Mine moved 2.5× on identical infrastructure purely from how much context the router loaded.

• Savings nobody is hunting. An 80% reduction does not announce itself. It shows up as a new row in a table you were already generating.

📌 And it inverts the objection. The reflex is that evaluation is overhead. The harness I built to answer a latency complaint is the same harness that found an 80% cost reduction on a system I had already optimised. Evaluation is not the tax you pay for shipping AI. It is the instrument that finds the money.

What I would tell a platform team

1. Precision is the goal; cost and latency are bars. When a model disappoints, first ask which one it failed — you fix those differently.

2. Price per completed answer, not per million tokens. Nobody consumes a million tokens, and a failed request still bills.

3. Put a correctness metric in the comparison. "Figures kept" was invented for this problem and it is the only thing that stopped me shipping a model that answers fast and leaves the numbers out.

4. Warm up, take medians, count failures separately. All three exist because the version without them handed me a confident wrong answer.

5. ⭐ Keep the harness and re-run it. Continuous integration for code, continuous evaluation for models — because the right answer moves without asking your permission.

The 18¢ piece found the cost bar breached by rework, because verification was missing. This one finds the latency bar breached by capability the task never asked for. Same goal, different bar — and neither is safely reclaimed without the same instrument underneath.

One site, one workload, five runs per model. If you have measured this at real scale and got something different, I want to hear it.

This is #1 in The Economics of Shipping AI. It measures the cost bar. #2 measures the latency bar — six models on one production system, and the one I was running turned out to be the most expensive and the slowest of them.

Written as the companion to my lightning talk at ClawCamp SF, Saturday 15 August 2026. If you just scanned the code from the room: this is the whole argument, with the sources.

Slides from the talk: https://xteamos.exponentialos.io/deck/

A survey of 2,444 companies put a number on something most engineering teams already feel.

For every dollar spent on AI tokens: 44 cents goes to fixing bugs the AI created. 27 cents to rewriting its code. 11 cents to review and merge delays.

Eighteen cents reaches production.

That number went around as “82% of your AI spend is wasted,” and I want to deal with the objection before I use it, because the objection is good.

The number is directionally right and numerically dubious

The study comes from Entelligence, a company that sells tooling for the exact problem the statistic describes. The methodology is self-reported. The 2,444 companies likely include free-tier accounts. A forensic teardown of the claim landed on the right verdict: directionally right, numerically dubious.

The teardown’s sharpest point is the one nobody quoting the stat mentions: pre-AI software development already spent 75–80% of effort on maintenance and debugging. So 82% isn’t a shocking new tax. It’s roughly the historical baseline wearing a new hat.

Which means the interesting question isn’t “how much is wasted.” It’s what actually changed.

What changed: generation outran review

Rework has always been most of the work. What’s new is the ratio between how fast you can produce code and how fast anyone can check it.

CodeRabbit measures AI-generated changes carrying roughly 1.7× more issues than human-written ones. Lightrun’s 2026 report found 43% of AI-generated code still requires manual debugging in production — after it passed quality checks.

Sit with that second one. Not “before review caught it.” After.

So the bottleneck was never cost per token. It’s review throughput, and every model upgrade makes the imbalance worse, because generation scales and human review does not.

That reframes what a quality gate is. A gate isn’t a tax on velocity. A gate is the only thing that raises the share of your spend that reaches the product. Yield, not savings.

Three things are quietly eating that yield. The first you can fix before you write a line, and the last is the one nobody says out loud.

Leak zero: you specified it wrong

Before routing and before context, there is a cheaper failure. The most expensive bug is the one you specified wrong, because everything downstream then executes the wrong thing faithfully and on budget.

You know what you want. Getting it into words precise enough for a model to act on is the actual work, and it is where the time and the tokens go. A vague request does not fail loudly. It returns something plausible, you read it, you realise it is not what you meant, and you ask again in slightly different words. Do that three times and you have paid for four answers to get one. A meaningful share of the 44 cents starts there, before a single routing or context decision.

Neither of the other two fixes helps you here. You can send a badly specified task to the cheapest capable model, with a perfectly curated window, and still get back code you have to rewrite.

The fix is unglamorous. State the outcome rather than the steps: what must be true when it is done, who it is for, and what would make it wrong. Then leave the how to the model. That is most of it.

I automated this for myself, because I was never going to do it by hand on every request. The prompt gets rewritten before it runs and I see both versions, so the sharpening happens inside the ten seconds I was going to spend anyway rather than in a fourth attempt.

Leak one: you route every task as if it were the hardest one

You get access to a frontier model. It’s good at everything, so you send it everything — file reads, git operations, deploys, browser automation, boilerplate tests, and the two decisions a day that actually need judgment.

Those are not the same task. You are paying judgment prices for clerical work, and you hit the rate ceiling by mid-afternoon because you spent the budget on git status.

The fix is routing by task class rather than by habit:

1) Deterministic work — deploys, git, file operations — goes to a small fast model. On current public pricing that’s roughly 20× cheaper than the frontier tier.

2) Browser automation and large reads go to a cheap high-context model, roughly 50× cheaper. This one matters more than it sounds: a single page snapshot can dump an entire DOM into your context, and you pay for those tokens on every subsequent turn.

3) Code generation goes to a code-specialised model.

4) The expensive model does one thing: judgment. Specs, gates, synthesis. That’s it.

None of that is controversial. It’s the second leak that people argue with.

Leak two: a bigger context window makes your agent worse

Everyone has been told to start a new session when things go sideways. Almost nobody explains why.

It’s a cure prescribed without a diagnosis, and once you have the diagnosis, the advice turns out to be the crudest possible intervention.

Why a fresh session feels smarter

Not because it’s emptier. Because the wrong turns are gone.

Every dead end you explored is still sitting in that window. The three incorrect things you told it about your schema two hours ago are still there, still being conditioned on. A fresh session doesn’t give the model more room to think — it removes the accumulated wrong answers.

That distinction kills the naive fix. Compressing by volume doesn’t help if you compress the wrong turns along with the right ones. You end up with a tidy summary of your own mistakes.

The research says it’s worse than dilution

Two findings, both well-established, both counterintuitive.

Position matters, badly. Liu et al., Lost in the Middle: How Language Models Use Long Contexts (Stanford, published in TACL), found performance is highest when relevant information sits at the beginning or end of the context and degrades significantly when the model must retrieve from the middle. Put the important thing in the middle of a long window and you have hidden it.

Irrelevant context isn’t neutral — it competes. Chroma’s context-rot work and the GSM-DC benchmark both find that models are meaningfully degraded by distractors, and that semantic similarity drives decay more than length does. A distractor that looks like the answer costs you far more than one that obviously doesn’t. A single distractor already degrades baseline; several compound it.

And then the finding that should change how you think about this entirely:

Models performed better on shuffled context than on logically coherent context — across 18 models. Coherent documents share terminology and structure, which makes them better distractors.

Read that again, because it inverts the instinct everyone has. The more relevant-looking the material you stuff into the window, the more damage it does. “Just give it all the related files, it’s all connected” is precisely the wrong move.

“It fits” is not the same as “it helps.”

The architecture: the smallest thing that does the job

The other two leaks have the same shape, and the same fix.

Sharp focus. Keep the working context deliberately small. Not as a limitation you’re working around — as a design choice.

Long-term memory. Everything else lives in an index outside the window. The agent doesn’t need to hold it. It needs to know where it is.

Just-in-time injection. Hydrate only the relevant slice, at the moment the task needs it. When the session ends, distil what was learned back into the index.

That’s what I run daily. The agent doesn’t carry everything. It carries the right thing at the right moment and knows where the rest lives.

Which makes “start a new session” look like what it is: amputation. It works. It also throws away everything you learned getting there. That’s only a rational trade if you have no memory layer — which, for most people giving the advice, is true.

Two better techniques:

Self-compress on your own schedule. The critical word is your. Auto-compaction fires when the system hits a limit, which is precisely the moment you have least control over what survives. Compress deliberately, at a natural boundary, and you choose what carries forward — by relevance, not by volume.

Write the handoff as insurance. You don’t get to pick when a session dies. This is the case you can’t schedule, which is exactly why it’s load-bearing.

Both convert a reset from a loss into a checkpoint.

The gates, and why they’re the yield story

Cheap routing and a small context window let you generate faster. That’s only an improvement if what you generate is right. Otherwise you’ve optimised your way into producing defects more efficiently.

So: gates. Acceptance criteria and evals fixed before code is written. Ships only if it beats baseline.

If you take one gate from this piece, take this one.

Never let the model that wrote the code be the model that reviews it.

Not a different prompt. Not a different persona. A different family. Claude reviewing Claude shares the training, so it shares the blind spot — and it will confidently approve its own mistake.

I didn’t want to believe that on vibes, so I tested it: 50 artifacts seeded with known flaws from a taxonomy of five failure modes, four review conditions, 240 review runs, hypothesis registered before a single run so I couldn’t move the goalposts afterward.

But the production evidence is blunter than the experiment.

A cross-family review panel caught a data-integrity bug and a silent cost-tracking regression that the CI gates — including 100% SonarCloud coverage — had waved through. Perfect coverage. Static analysis green. Two real defects shipped anyway, one of them quietly burning money.

That is the Lightrun 43% statistic happening in a single repository.

Two more from the same period:

A single judge passed an attribution defect that the cross-family panel flagged RED at 97% confidence. One event keyed a raw session-row id while its pair used a different identity, so conversion credit would never have joined back to the referrer. Same code, same moment — one reviewer missed it, a panel of different families caught it.

One model caught a corrupt-state bug that two models from the same other family both missed — corrupt state silently wiping configuration to defaults.

I’m not going to give you a catch rate. I don’t have a denominator, and a made-up percentage would undo everything above. What I have is a dated record of gates catching defects that coverage, unit tests, and single-model review all passed.

The tax nobody prices

Rework doesn’t only burn tokens. It burns the thing you can’t buy back.

Debugging plausible-looking code you didn’t write is a genuinely harder cognitive task than debugging your own. You have no mental model to fall back on, so you reconstruct intent from scratch. Every defect pulls someone out of build mode, and the context switch costs more than the fix.

And then the compounding one: every defect that ships through your gates teaches the developer to stop trusting the output. Once that happens, they re-read everything — and you have paid for speed and lost it.

That isn’t a soft cost. It’s the mechanism by which the entire investment decays, and it appears in none of the studies above, because nobody is measuring it.

What to change on Monday

Zero. Before you send your next non-trivial request, write one sentence saying what must be true when it is done and what would make it wrong. Notice how often you cannot, because that is the answer.

One. Take the three most repetitive things you send to a frontier model this week and route them to a cheap one. Measure the difference. You will be annoyed at how much you were spending.

Two. Take whatever you stuff into context by default and move it behind a retrieval step. Inject it when the task needs it, not before. Then add one review pass from a different model family before anything ships.

The model isn’t your bottleneck. Your request and your routing are.

What I built, and what it costs you

Every fix in this piece is implemented in one place, and I did not write the piece and then go looking for a product to attach to it. The product came first.

Leak one, leak two and the gate are the Development pack in xTeamOS: cost routing, staged quality gates, and review by a model family that did not write the code. It is live and running in production today, because it is the pipeline that ships my own work.

Leak zero is the Ideation pack: fixing the thinking before the code. That one is still coming, but the prompt-sharpening half of it is available today as Co-Dialectic, which is free and open source.

Early access to xTeamOS is free. Everyone who joins during beta gets 50 percent off the subscription for their first year when it reaches production.

What the subscription buys is maintenance you do not have to do. Bug fixes and upgrades as the model landscape shifts underneath all of us, which it does constantly. You focus on shipping your product; the research and the plumbing stay my problem.

The cost difference is the part worth reading twice. You bring your own LLM subscriptions. There is no API key to fund, unlike tools that meter you through their key and bill you for tokens on top of a seat. You are already paying for those subscriptions. Use them.

And the direction of travel: support for low-cost open models, on the order of five times cheaper, for the work that does not need frontier judgment. Which is this entire argument applied to the tool itself.

Four things, in order of how little they cost you

Install Co-Dialectic. Free, open source, about a minute: codi.exponentialos.io. It is leak zero, fixed, today.

Join the xTeamOS beta. Also free, seats are limited, and beta members keep 50 percent off the first year: exponentialos.io.

Follow Exponential OS on LinkedIn for the build as it happens: linkedin.com/company/exponentialos.

Share this with whoever on your team is burning the other 82 cents. That is the one I cannot do myself.

This is Paper 1 in the Reliable AI Systems series, which argues that AI systems must be governed at multiple layers simultaneously to be reliable in deployment. Each piece takes one layer and contributes a primitive. The companion piece, Defense in Depth for AI Agents, covers the operational layer: structured evaluation pipelines, drift monitoring, and guardrail-as-architecture.

I’m Anand Vallamsetla — Exponential OS, ex-Google, ex-AI Fund (Andrew Ng’s venture studio). I build the systems that build AI products.

Sources

• Entelligence AI — token spend breakdown across 2,444 companies — https://research.entelligence.ai/

• Lightrun — 2026 State of AI-Powered Engineering — 43% of AI code needs manual debugging after passing quality checks

• CodeRabbit — AI-generated changes carry ~1.7× more issues than human-written

• Jake Cuthbertson — The Rework Tax, forensic teardown of the 82% claim — https://jakecuth.com/work/ai-rework-lab/

• Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, Liang — Lost in the Middle: How Language Models Use Long Contexts, TACL 2024 — https://arxiv.org/abs/2307.03172

• How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark (GSM-DC) — https://arxiv.org/abs/2505.18761

• Chroma — context rot research on distractor semantics and shuffled-vs-coherent haystacks

That gap is the whole story. Ranking on Google and being citable by AI are not the same problem, and treating them as one is why so many sites have gone quiet in AI answers without noticing.

Here's what's actually happening under the hood.

An AI answer comes from one of two places. Either the model answers from memory — what it absorbed in training, which you can't change and which is months stale — or it runs a live retrieval, fetches pages, and cites them. Only the second path is one you can influence. And that path has two gates almost nobody checks.

Gate 1: Can the crawler even fetch you?

The retrieval bots behind ChatGPT, Claude, and Perplexity — OAI-SearchBot, ClaudeBot, PerplexityBot — do not run JavaScript. A large-scale study of roughly a billion requests found none of them render JS. So if your site builds its content in the browser (any SPA — React, Vue, most modern stacks), the crawler sees an almost-empty shell where a human sees a full page. You can rank on Google, which does render JS, and be a blank page to ChatGPT.

The other half of Gate 1 is your own security layer. A WAF or Cloudflare rule that challenges "bot-like" traffic will 403 the AI crawler while serving humans normally. You never see it. You just quietly disappear.

Here's the ten-second test. Run this on your own site:

curl -A "ClaudeBot/1.0 (+https://www.anthropic.com/claude-bot)" https://yoursite.com

If you get your real content back, good. If you get an empty shell, a redirect loop, or a 403 — that's why you're not cited, and no amount of content strategy fixes it until you fix this.

Gate 2: Once fetched, the index decides — and every engine's index is different

I ran one experiment that made this vivid. I asked four assistants the same open question — "best API to extract structured data from PDFs" — each on its own search index.

• ChatGPT (Bing index): surfaced exact-match-domain microsites — frompdf.dev, transpdf.ai, extractocr.com. When I actually fetched frompdf.dev, it returned zero bytes — no readable content at all — yet it was cited the most. These sites have no measurable traffic and zero presence on Reddit. They won on one thing: their domain name is the query.

• Gemini (Google index): Google Document AI, AWS Textract, Azure, LlamaParse, Unstructured — plus LandingAI, a legitimate, well-built player with a fraction of the incumbents' traffic.

• Claude (Brave index): cloud incumbents and real tools; Parseur, Klippa, LlamaParse.

Same question. Four different worlds. On Bing/ChatGPT, keyword-stuffed domains with no audience beat a readable, funded product. On Google and Brave, legitimacy won. "Getting cited by AI" isn't one target — it's four, and they reward different things.

There's a concentration story on top of this. One synthesis of 680M+ AI citations (Everything-PR / 5WPR, 2026) found the top 15 domains capture ~68% of all citations, with Reddit alone near 40% and #1 on every engine. (It's a third-party aggregation, not an audited study — treat it as directional.) Much of AI citation flows through a handful of high-trust intermediaries you don't own — a different fight from making your own site citable.

What doesn't work, despite the hype

llms.txt. One audit of 1,500 sites found 0.2% use it, and real crawler consumption today is near zero. Schema/JSON-LD helps a parser understand you but doesn't get you retrieved. Both are fine hygiene; neither is the lever people think it is.

So, in order of impact

1. Make sure AI crawlers can fetch you — server-render or pre-generate your main content, and stop challenging verified bots at the edge. (The curl test above.)

2. Put concrete, quotable facts in the first few hundred words, not boilerplate.

3. Optimize per engine, not in general — the index that feeds your buyers' assistant is the one that matters.

4. llms.txt last, if at all.

None of this requires a rebrand. It requires knowing what an AI agent actually sees when it reads you — which, it turns out, is almost never what you see in a browser.

(I got tired of running that curl test by hand, so I built a free scanner that fetches your page as ChatGPT and Claude's crawlers do and shows the exact gaps — no signup: getagentview.com. But the curl one-liner above will get you 80% of the way for free.)

Caveats, because they matter: the four-engine test is a single snapshot, not a controlled study — results drift week to week. The citation-share figures are third-party aggregations. And correlation isn't causation — reachability tracks closely with citation in my testing, but the clean proof is a before/after on a site you actually fix.