AI Search Readiness Checker: Can AI Search Find Your Site in 2026?

Enter any URL above. The checker reads the site's robots.txt and reports, for 13 AI crawlers, whether the exact URL you entered is allowed or blocked. It also checks for /llms.txt and lists the schema.org types in the page's JSON-LD. It takes a few seconds and needs no sign-up.

The rest of this page explains what each result means, which ones matter for being cited by ChatGPT, Claude, Perplexity and Google's AI features, and which ones are noise.

How to read your AI search readiness checker results

The result has three blocks. Here is what each common outcome means and what, if anything, to do about it.

"No robots.txt found, so every crawler is allowed by default." Nothing is blocked by robots rules. That is fine for AI visibility, but publish a robots.txt anyway so you can list your sitemap in it and make deliberate choices later. A site with no robots.txt also tells a reviewer that nobody has looked at crawling.

All 13 crawlers allowed. Your robots.txt does not stand in the way of any AI product. If you still never appear in AI answers, the cause is further down the pipeline: a firewall rule, JavaScript-rendered content, or pages that do not answer questions directly.

Search crawlers blocked, training crawlers blocked. Usually the result of a "block all AI" template or a plugin setting. If you want to be cited in ChatGPT, Claude or Perplexity answers, allow OAI-SearchBot, Claude-SearchBot and PerplexityBot and keep the training bots blocked. This is the most common fix we see.

Training crawlers blocked, search crawlers allowed. A deliberate and sensible setup for most publishers: your content is not used for model training, but it can still be found and cited.

Only some paths blocked. The checker tests the exact URL you enter. A rule like Disallow: /blog/ affects blog posts but not the homepage, so run the check on one URL from each important section.

ChatGPT-User or Perplexity-User shows "Blocked". The rule is in your robots.txt, but both companies say their user-triggered fetchers may not follow it. Treat the result as your stated preference, not as a guarantee.

llms.txt "Not published". Not a problem. Google says the file has no effect on Search, and no AI search company has confirmed using it. Add one only if it costs you nothing.

Schema types: "None found". The page has no JSON-LD. On the homepage, add Organization and WebSite; on content and product pages, add the type that matches. Types written in microdata or RDFa are not detected, because the checker reads JSON-LD only.

An error instead of results. The site did not respond, redirected to a blocked address, or refused our request. If your firewall blocks unknown user agents, it is probably blocking some AI crawlers too.

What AI search readiness means in 2026

AI search readiness is whether AI-powered search products can find your pages, read them, and choose to cite them in an answer. It is the AI-era version of "can Google crawl and index this page?", with one extra wrinkle: there is no single AI crawler. Each company runs several bots for different jobs, and robots.txt rules apply to each one separately.

It became a real business question in 2025 and 2026 for three reasons.

AI answers now carry measurable traffic. In June 2026 Google added generative AI performance reports to Search Console, showing impressions in AI Overviews and AI Mode next to normal results. ChatGPT search, Perplexity and Claude all show cited sources with links. A page that is blocked from these products is simply absent from a growing share of answers.

Many sites block AI crawlers without knowing it. In July 2025 Cloudflare began blocking AI training crawlers by default for new domains, and from 15 September 2026 new Cloudflare domains block bots classed as Training or Agent on pages that show ads. Security plugins, hosting firewalls and copy-pasted robots.txt templates do the same. Site owners who meant to stop model training often blocked AI search too.

The bots split up. OpenAI separates GPTBot (training), OAI-SearchBot (ChatGPT search) and ChatGPT-User (fetches a page when someone asks). Anthropic documented three equivalents in February 2026: ClaudeBot, Claude-SearchBot and Claude-User. Blocking "the AI bot" no longer means anything; you have to decide per job.

How AI search systems find and cite a page

The pipeline has four stages, and a site can fail at any of them:

1. Crawl. A search crawler (OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot) fetches the page, if robots.txt and the site's firewall let it.
2. Read. The crawler parses the HTML. Most AI crawlers do not run JavaScript. Vercel's analysis of crawler traffic in late 2024 found that OpenAI's and Anthropic's crawlers fetched JavaScript files but did not execute them, so content injected client-side is invisible to them. Google's AI features use Googlebot's rendering, which does run JavaScript.
3. Retrieve. When someone asks a question, the system searches its index (or the live web, via a user-triggered fetch) for passages that answer it.
4. Cite. It quotes or summarises the passages that answer the question most directly, with a link.

Readiness checks cover stages 1 and 2. Being chosen at stages 3 and 4 depends on the content itself, which is why a page can pass every check here and still not be cited.

How to check AI search readiness in 7 steps

The tool at the top of this page automates steps 1 to 3. The rest take a few minutes by hand.

1. Run the checker on your homepage and on one important inner page, because robots.txt rules often differ by path.
2. Read the crawler results: confirm the search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) are allowed if you want to be cited in those products.
3. Check whether /llms.txt exists and whether the schema types match the page (Organization on the homepage, Article or Product on inner pages).
4. Open your CDN or firewall settings (Cloudflare, Vercel, a WordPress security plugin) and confirm AI crawlers are not blocked at the network level, which robots.txt cannot show.
5. View the page with JavaScript turned off and confirm the main heading and the opening answer appear in the raw HTML.
6. Ask ChatGPT search, Perplexity and Claude a question your page answers and note whether you are cited and which pages are.
7. Check Search Console's generative AI performance report monthly to see whether impressions in AI Overviews and AI Mode are growing.

Step 4 catches the most surprises. A robots.txt can allow every bot while the CDN returns 403 to all of them. The checker can only read robots.txt; it cannot see a firewall rule.

AI crawler access: which bots to allow

Each AI company runs bots with different purposes, and they treat robots.txt differently. This is the current picture from each company's own documentation:

Two points trip people up. First, Google-Extended does not control AI Overviews or AI Mode; those use normal Googlebot crawling, so the only way to leave them is to leave Google Search. Second, the user-triggered fetchers sit outside robots.txt for OpenAI and Perplexity, so robots.txt is not a complete block.

The full guide to reading these results, including the robots.txt patterns that block search crawlers by accident, is the AI crawler access checker page. Each major bot also has its own page with copy-paste rules: GPTBot robots.txt, ClaudeBot robots.txt and PerplexityBot robots.txt.

A robots.txt that allows search and refuses training looks like this:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: OAI-SearchBot
User-agent: Claude-SearchBot
User-agent: PerplexityBot
Allow: /

User-agent: *
Allow: /
Disallow: /admin/

Remember that a crawler obeys only the most specific group that names it. If you give OAI-SearchBot its own group, it ignores the Disallow: /admin/ under User-agent: * unless you repeat it there.

llms.txt: what it is and what it does not do

llms.txt is a proposed file, published at the root of a site, that lists its most useful pages in Markdown for language models to read. Jeremy Howard of Answer.AI proposed it in September 2024. It is easy to make and many documentation sites publish one.

What it does not do is affect Google. Google's Gary Illyes said in July 2025 that Google does not support it, and Google's AI optimisation guide, updated on 15 June 2026, states that llms.txt has no effect on Search, including AI features. No other major AI search company has confirmed using it for ranking or citation either.

That makes it a low-cost, low-certainty addition: worth five minutes if your site has documentation or a product catalogue an agent might navigate, not worth prioritising over crawler access or content. The checker reports whether you have one. The llms.txt checker guide covers the format and validation, and does llms.txt work summarises the evidence so far.

Structured data for AI search

Schema markup does not make a page rank in AI answers. What it does is state facts unambiguously: this is an organisation called X with this logo and these profiles; this is a product with this price; this article was updated on this date. Systems that build knowledge about entities can read those facts without guessing from layout.

The checker lists every schema.org type it finds in the page's JSON-LD. The useful baseline:

• Organization (or LocalBusiness) on the homepage, with name, url, logo and sameAs links to your official profiles.
• WebSite on the homepage.
• Article or WebPage with dateModified on content pages.
• Product with offers on product pages, and BreadcrumbList on anything below the top level.

FAQPage markup no longer produces FAQ rich results in Google (they ended on 7 May 2026), but it still describes question-and-answer content cleanly. Use it only where the questions and answers are visible on the page. The full guide, with examples per page type, is schema markup for AI search.

Content that AI search actually cites

Access and markup get a page considered. Content decides whether it is quoted. Pages that get cited in AI answers share a few traits you can check on your own pages:

• The answer comes first. A direct one- to three-sentence answer to the question directly under the heading, before background.
• Passages stand alone. Each section makes sense if lifted out of the page, with the subject named rather than "it" or "this".
• Specific facts. Numbers, dates, names and limits. An answer engine choosing between "prices vary" and "$29 a month for 2,000 pages" quotes the second.
• Clear structure. Steps in numbered lists, comparisons in tables, one topic per heading.
• Signs of a real source. An author or organisation, an updated date, links to primary sources.

Google's own guidance for AI features says the same thing in plainer terms: normal SEO and useful content are what count. The practical version, with before-and-after rewrites, is in how to get cited by ChatGPT.

AI search readiness signals compared

Not every signal deserves equal attention. This table ranks them by how much evidence there is that they affect AI visibility.

Common AI readiness mistakes

Blocking every AI bot to stop training. Wrong: User-agent: * plus a list of every AI crawler set to Disallow: /, copied from a "block AI" template. Right: block GPTBot and ClaudeBot if you object to training, and allow OAI-SearchBot, Claude-SearchBot and PerplexityBot so you still appear in AI answers.

Using Google-Extended to leave AI Overviews. Wrong: adding User-agent: Google-Extended / Disallow: / and expecting AI Overviews to stop quoting you. Right: knowing that AI Overviews use Googlebot, so Google-Extended only affects Gemini training and grounding. There is no robots.txt rule that removes a page from AI Overviews but keeps it in Search.

Trusting robots.txt alone. Wrong: "Our robots.txt allows everything, so AI search can see us." Right: also checking Cloudflare's AI bot settings, the WAF and any security plugin, because a network-level block overrides robots.txt and leaves no trace in the file.

Putting the content in JavaScript. Wrong: a single-page app that ships an empty <div id="root"> and renders the text in the browser. Right: server-render or pre-render the title, H1 and main text, so crawlers that don't execute JavaScript still read the page.

Tools for AI search readiness

This checker covers crawler rules, llms.txt and schema types for any public URL. It is free and has a limit of 20 checks an hour per visitor.

Google Search Console: the generative AI performance report shows AI Overviews and AI Mode impressions; URL Inspection shows the rendered HTML Googlebot saw.

Server logs or your CDN's bot analytics show which AI crawlers actually visit and what status they receive. Cloudflare's AI Crawl Control lists them by name.

OpenAI's, Anthropic's and Perplexity's crawler documentation are the primary sources for bot names and behaviour, and they change; check them when you edit robots.txt.

Seotific's full audit goes beyond access. It checks whether the page is worth citing: the opening answer, keyword match in the title and H1, headings, lists and tables, schema validity and whether the main content is in the raw HTML. It also checks Organization details on the homepage and flags AI crawler blocks as part of the same report. For the wider audit beyond AI search, see the SEO audit checklist.

Frequently asked questions