Can AI crawlers read your site?
Paste a URL. We fetch it the way GPTBot, ClaudeBot, PerplexityBot and Google-Extended do - no JavaScript - and run the 13 on-page checks from the audit, in seconds.
The result is not stored and no engine is asked. Ten checks per network per day.
Results
| Status | Check | Detail |
|---|
Brand-name consistency, the 14th check in a report, needs the product identified first; the snapshot does that.
See whether the engines actually recommend you → free snapshot
What does the AI crawler check test?
Thirteen deterministic checks on your homepage and its robots.txt, llms.txt and sitemap.xml, each pass, warn or fail with the value we found. They are the same code the audit runs; nothing here asks a language model.
| Check | What it means |
|---|---|
| Homepage reachable | HTTP 200 on the URL you pasted. Anything else and no crawler gets further; the check stops here. |
| AI crawlers allowed in robots.txt | Fails when a rule blocks GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended or CCBot. Warns when there is no robots.txt at all: crawlers default to allowed, but you cannot express intent. |
| Content visible without JavaScript | Fails under 300 characters of text in the raw HTML; warns when more than 40% is hidden by inline styles until scripts run. Crawlers do not execute JavaScript. The most common failed check in our audits. |
| Title and H1 | A non-empty title and at least one H1: what an engine quotes as the page's name. |
| Meta description | Present on the homepage; the one-line summary engines fall back to. |
| Canonical URL | Present, so one address gets the citation rather than several near-copies splitting it. |
| llms.txt present | A plain-text /llms.txt with a 200 status. Cheap to publish; not a lever on its own (why). |
| sitemap.xml present | A /sitemap.xml that parses. Bing and Brave, the indexes behind ChatGPT and Claude, find new pages through it. |
| JSON-LD Organization / WebSite | Any of those types on the homepage, so engines do not have to guess who you are. |
| JSON-LD Product / Offer / SoftwareApplication | Product or pricing schema on the homepage. |
| Answer-first FAQ block | Question-and-answer structure on the page: engines lift short direct answers from these. |
| dateModified / modified_time signal | A freshness signal in the HTML; engines prefer visibly current pages. |
| Pricing visible in HTML | Price strings in the raw text, so an engine answering "how much does it cost" has something to quote. |
Which crawlers does it look for, and which engine does each feed?
- ChatGPT:
OAI-SearchBotbuilds the search results,ChatGPT-Userfetches a page when a person asks about it,GPTBotis the training crawler. Guide: ChatGPT. - Claude:
Claude-SearchBotbuilds the index,ClaudeBotis training. Rules for one do not cover the other. Guide: Claude. - Perplexity:
PerplexityBot. Guide: Perplexity. - Gemini:
Google-Extendedcontrols training use only; whether Gemini can cite you depends on ordinary Googlebot indexing. Guide: Gemini. - Common Crawl:
CCBot, the open crawl several models train on.
Does blocking GPTBot remove me from ChatGPT's answers?
Not directly: search results come from OAI-SearchBot and Bing, not from GPTBot. In practice, though, sites that block GPTBot are cited about 140 times less often than sites that allow it, and a small site cannot make that up through syndication the way a large publisher can. Decide about the training crawlers separately, but let the search crawlers in. The checklist lists the set to allow.
Why does "content without JavaScript" matter so much?
Every crawler above reads the HTML your server sends and moves on; none of them runs your front-end. If the answer to a buyer's question is rendered by a script, the engine sees an empty shell and cites whoever wrote the same thing in plain HTML. Fetch your page with curl: what comes back is what the engines see. Server-side rendering or pre-rendering fixes it for good.
What does a passing check not tell you?
Whether any engine actually recommends you. Being readable is the entry ticket; the levers that move citations are third-party mentions, original data and answer-first pages (ranked by evidence). The free snapshot asks ChatGPT, Claude, Gemini and Perplexity five of your buyers' questions and shows who each engine links to - you, a rival, or nobody.
What do you keep from a check?
Not the result: it is rendered in your browser and never written to a database. We keep a hash of your network address for 48 hours to enforce the ten-per-day limit, and one line in our ops log with the hostname checked, swept after 30 days. Details are in the privacy notice.