Guide

AI Readiness: How to Get Cited by ChatGPT and Perplexity

AI assistants only recommend sites they can crawl, parse and trust. Those are three separate problems, and most sites fail at least one without knowing.

By WebDoctor editors·16 July 2026·3 min read


A site is AI-ready when answer engines like ChatGPT, Perplexity and Google's AI results can crawl it, understand it and cite it. Those are three separate layers: access, controlled by robots.txt; understanding, driven by structured data and metadata; and guidance, provided by llms.txt. Each layer fails independently, and most sites have at least one gap.

Why this became its own discipline

A growing share of questions that used to be Google searches now get answered inside an AI assistant, with a handful of cited sources. Being one of those citations is the new position one. Classic SEO still matters, because answer engines lean on search indexes to find candidates, but the overlap is partial: a site can rank fine in blue links and still be invisible to AI answers because it blocked a crawler two years ago during the training-data debates.

Layer 1: access. Who can read your site?

Every major AI system crawls with a named user agent, and your robots.txt decides bot by bot. The critical distinction is what each bot is for. Blocking a training bot only affects future model weights; blocking a search bot removes you from that engine's answers today.

CrawlerOperatorBlocking it means
GPTBot, OAI-SearchBotOpenAIChatGPT cannot read or cite you
ClaudeBotAnthropicClaude cannot read or cite you
PerplexityBotPerplexityAbsent from Perplexity answers
Google-ExtendedGoogleOnly Gemini training; AI Overviews unaffected
CCBotCommon CrawlOut of many training datasets; no search impact

The most common self-inflicted wound: a blanket "block all AI bots" robots.txt added in 2023 as a statement about training data, still in place now that the same user agents power search products. Decide per bot, based on what it does today.

Layer 2: understanding. Can machines parse what you mean?

Answer engines do not read pages the way people do. The signals they parse most reliably:

  • JSON-LD structured data. Schema.org markup (Organization, FAQPage, Article, Product) is unambiguous machine-readable fact, and it is what gets extracted into answers most cleanly. FAQ markup is especially effective because it is literally in question-and-answer form.
  • Honest metadata. A specific title tag, a real meta description, a canonical URL and one clear H1 per page. AI snippets fall back to arbitrary page text when these are missing.
  • Answer-shaped content. Pages that state the answer in the first paragraph get quoted; pages that bury it under 800 words of preamble get skipped. Comparison tables and FAQ sections are extraction-friendly by construction.

Layer 3: guidance. Did you leave a map?

An llms.txt file gives models a curated index of your site in markdown. It is the newest and least standardized layer, which is why it is layer three and not layer one: do it after access and understanding are right, not instead of them. It takes minutes with a generator and closes the loop.

Fix-first checklist

  1. Run the AI readiness check on your domain. It scores all three layers in about ten seconds.
  2. Unblock the AI search crawlers you want citing you. This is the highest-impact single change, and it is one robots.txt edit.
  3. Add JSON-LD to your key page types, starting with Organization on the homepage and FAQPage wherever you genuinely answer questions.
  4. Fill the metadata gaps the check flags: titles, descriptions, canonicals.
  5. Confirm your sitemap is valid and declared in robots.txt; answer engines use search indexes to find you.
  6. Add llms.txt last, and re-run the check to confirm everything reads green.

What AI readiness is not

It is not a trick, and there is no secret meta tag. Systems built to answer questions reward the same things good technical SEO always rewarded: reachable pages, unambiguous structure, content that actually answers something. The difference is enforcement. A human visitor tolerates a missing meta description; an extraction pipeline just moves to the next source. AI readiness is technical hygiene with the tolerance turned down.

AI searchTechnical SEO

Frequently asked questions

Should I block AI crawlers to protect my content?
It is a real trade-off, but know the price: blocking search-connected bots like GPTBot or PerplexityBot removes you from those engines' answers entirely. If your business benefits from being discovered and cited, blanket blocks work against you. Blocking training-only bots like Google-Extended or CCBot is a much narrower statement with no search cost.
Do AI assistants actually send traffic?
Yes, and it converts well: visitors arriving from a cited answer already chose you over the summary. Volumes are still small next to Google for most sites, but they are the fastest-growing referral class, and citation positions are being settled now, while competition is thin.
Does ranking well in Google mean I am AI-ready?
Not necessarily. Answer engines use search indexes to find candidate pages, so ranking helps discovery. But a robots.txt block, missing structured data or unparseable pages can keep a well-ranking site out of AI answers. The layers are related and fail independently, which is why checking each one matters.
What single change improves AI visibility most?
If any AI search crawler is blocked in robots.txt, unblocking it, because a blocked crawler means zero visibility in that engine no matter how good everything else is. If access is already open, adding FAQPage and Organization JSON-LD typically moves the needle next.
How often should I re-check AI readiness?
After any robots.txt change, site migration or template redesign, and quarterly otherwise. The crawler landscape is moving fast; new bots appear and old rules quietly become wrong, like training-era blocks now hitting search bots.

Run a check

Keep reading