AI Readiness: How to Get Cited by ChatGPT and Perplexity
AI assistants only recommend sites they can crawl, parse and trust. Those are three separate problems, and most sites fail at least one without knowing.
A site is AI-ready when answer engines like ChatGPT, Perplexity and Google's AI results can crawl it, understand it and cite it. Those are three separate layers: access, controlled by robots.txt; understanding, driven by structured data and metadata; and guidance, provided by llms.txt. Each layer fails independently, and most sites have at least one gap.
Why this became its own discipline
A growing share of questions that used to be Google searches now get answered inside an AI assistant, with a handful of cited sources. Being one of those citations is the new position one. Classic SEO still matters, because answer engines lean on search indexes to find candidates, but the overlap is partial: a site can rank fine in blue links and still be invisible to AI answers because it blocked a crawler two years ago during the training-data debates.
Layer 1: access. Who can read your site?
Every major AI system crawls with a named user agent, and your robots.txt decides bot by bot. The critical distinction is what each bot is for. Blocking a training bot only affects future model weights; blocking a search bot removes you from that engine's answers today.
| Crawler | Operator | Blocking it means |
|---|---|---|
GPTBot, OAI-SearchBot | OpenAI | ChatGPT cannot read or cite you |
ClaudeBot | Anthropic | Claude cannot read or cite you |
PerplexityBot | Perplexity | Absent from Perplexity answers |
Google-Extended | Only Gemini training; AI Overviews unaffected | |
CCBot | Common Crawl | Out of many training datasets; no search impact |
The most common self-inflicted wound: a blanket "block all AI bots" robots.txt added in 2023 as a statement about training data, still in place now that the same user agents power search products. Decide per bot, based on what it does today.
Layer 2: understanding. Can machines parse what you mean?
Answer engines do not read pages the way people do. The signals they parse most reliably:
- JSON-LD structured data. Schema.org markup (Organization, FAQPage, Article, Product) is unambiguous machine-readable fact, and it is what gets extracted into answers most cleanly. FAQ markup is especially effective because it is literally in question-and-answer form.
- Honest metadata. A specific title tag, a real meta description, a canonical URL and one clear H1 per page. AI snippets fall back to arbitrary page text when these are missing.
- Answer-shaped content. Pages that state the answer in the first paragraph get quoted; pages that bury it under 800 words of preamble get skipped. Comparison tables and FAQ sections are extraction-friendly by construction.
Layer 3: guidance. Did you leave a map?
An llms.txt file gives models a curated index of your site in markdown. It is the newest and least standardized layer, which is why it is layer three and not layer one: do it after access and understanding are right, not instead of them. It takes minutes with a generator and closes the loop.
Fix-first checklist
- Run the AI readiness check on your domain. It scores all three layers in about ten seconds.
- Unblock the AI search crawlers you want citing you. This is the highest-impact single change, and it is one robots.txt edit.
- Add JSON-LD to your key page types, starting with Organization on the homepage and FAQPage wherever you genuinely answer questions.
- Fill the metadata gaps the check flags: titles, descriptions, canonicals.
- Confirm your sitemap is valid and declared in robots.txt; answer engines use search indexes to find you.
- Add llms.txt last, and re-run the check to confirm everything reads green.
What AI readiness is not
It is not a trick, and there is no secret meta tag. Systems built to answer questions reward the same things good technical SEO always rewarded: reachable pages, unambiguous structure, content that actually answers something. The difference is enforcement. A human visitor tolerates a missing meta description; an extraction pipeline just moves to the next source. AI readiness is technical hygiene with the tolerance turned down.