Why your site is invisible to ChatGPT (even when Google loves you)
Ranking on Google and being citable by AI are not the same problem — and I asked four engines the same question to prove it.
An accountant asked a friend of mine a simple question: "If someone asks ChatGPT for an accountant near me, do I show up?" He had no idea. So he checked 15 small-business sites. Two showed up. Thirteen were invisible — most of them ranking perfectly well on Google.
That gap is the whole story. Ranking on Google and being citable by AI are not the same problem, and treating them as one is why so many sites have gone quiet in AI answers without noticing.
Here's what's actually happening under the hood.
An AI answer comes from one of two places. Either the model answers from memory — what it absorbed in training, which you can't change and which is months stale — or it runs a live retrieval, fetches pages, and cites them. Only the second path is one you can influence. And that path has two gates almost nobody checks.
Gate 1: Can the crawler even fetch you?
The retrieval bots behind ChatGPT, Claude, and Perplexity — OAI-SearchBot, ClaudeBot, PerplexityBot — do not run JavaScript. A large-scale study of roughly a billion requests found none of them render JS. So if your site builds its content in the browser (any SPA — React, Vue, most modern stacks), the crawler sees an almost-empty shell where a human sees a full page. You can rank on Google, which does render JS, and be a blank page to ChatGPT.
The other half of Gate 1 is your own security layer. A WAF or Cloudflare rule that challenges "bot-like" traffic will 403 the AI crawler while serving humans normally. You never see it. You just quietly disappear.
Here's the ten-second test. Run this on your own site:
curl -A "ClaudeBot/1.0 (+https://www.anthropic.com/claude-bot)" https://yoursite.comIf you get your real content back, good. If you get an empty shell, a redirect loop, or a 403 — that's why you're not cited, and no amount of content strategy fixes it until you fix this.
Gate 2: Once fetched, the index decides — and every engine's index is different
I ran one experiment that made this vivid. I asked four assistants the same open question — "best API to extract structured data from PDFs" — each on its own search index.
ChatGPT (Bing index): surfaced exact-match-domain microsites — frompdf.dev, transpdf.ai, extractocr.com. When I actually fetched frompdf.dev, it returned zero bytes — no readable content at all — yet it was cited the most. These sites have no measurable traffic and zero presence on Reddit. They won on one thing: their domain name is the query.
Gemini (Google index): Google Document AI, AWS Textract, Azure, LlamaParse, Unstructured — plus LandingAI, a legitimate, well-built player with a fraction of the incumbents' traffic.
Claude (Brave index): cloud incumbents and real tools; Parseur, Klippa, LlamaParse.
Same question. Four different worlds. On Bing/ChatGPT, keyword-stuffed domains with no audience beat a readable, funded product. On Google and Brave, legitimacy won. "Getting cited by AI" isn't one target — it's four, and they reward different things.
There's a concentration story on top of this. One synthesis of 680M+ AI citations (Everything-PR / 5WPR, 2026) found the top 15 domains capture ~68% of all citations, with Reddit alone near 40% and #1 on every engine. (It's a third-party aggregation, not an audited study — treat it as directional.) Much of AI citation flows through a handful of high-trust intermediaries you don't own — a different fight from making your own site citable.
What doesn't work, despite the hype
llms.txt. One audit of 1,500 sites found 0.2% use it, and real crawler consumption today is near zero. Schema/JSON-LD helps a parser understand you but doesn't get you retrieved. Both are fine hygiene; neither is the lever people think it is.
So, in order of impact
Make sure AI crawlers can fetch you — server-render or pre-generate your main content, and stop challenging verified bots at the edge. (The curl test above.)
Put concrete, quotable facts in the first few hundred words, not boilerplate.
Optimize per engine, not in general — the index that feeds your buyers' assistant is the one that matters.
llms.txt last, if at all.
None of this requires a rebrand. It requires knowing what an AI agent actually sees when it reads you — which, it turns out, is almost never what you see in a browser.
(I got tired of running that curl test by hand, so I built a free scanner that fetches your page as ChatGPT and Claude's crawlers do and shows the exact gaps — no signup: getagentview.com. But the curl one-liner above will get you 80% of the way for free.)
Caveats, because they matter: the four-engine test is a single snapshot, not a controlled study — results drift week to week. The citation-share figures are third-party aggregations. And correlation isn't causation — reachability tracks closely with citation in my testing, but the clean proof is a before/after on a site you actually fix.