Paste a page and see what AI crawlers get from it: twelve checks, each marked good, could be better or problem, then what to fix first. Free, no email.
Example: intentclose.com
IntentClose AI crawler check
https://intentclose.com/
9 good · 2 could be better · 0 problems
- AI crawlers allowed in robots.txtThe crawlers of OpenAI, Anthropic and Perplexity are all allowed, and so is Googlebot, whose rules Brave follows (Brave is the index Claude's web search reportedly uses).Good
- Your text is in the HTML AI crawlers readGPTBot received 5562 words; a browser shows 5562 words.Good
- ChatGPT can open the page when someone asksChatGPT's browsing agent (ChatGPT-User) received 5562 words.Good
- Title, description and main headingA 56-character title, a 159-character description and one H1.Good
- An llms.txt filePresent, with a title line, links and enough text for an AI tool to use.Good
- Structured data (JSON-LD)Found: Answer, Brand, DefinedRegion, FAQPage and 19 more.Good
- A sitemap crawlers can findFound, and declared in robots.txt.Good
- A last-updated dateFound: 2026-10-15T10:27:30+00:00.Good
- Ready to be quoted in an AI answerIt has 2 of the 5 things AI answers quote from. Missing: question-style headings, short self-contained paragraphs (20 to 90 words) and a named author.Could be better
- www and http:// lead to your pageBrowsers and crawlers are served normally. Plain script fetches are refused across the site, not just at www.intentclose.com: your main address answers one with HTTP 403 too. That is a bot filter at your host or CDN, not a redirect fault, and it usually lets verified search and AI crawlers through while stopping generic tools.Could be better
- What a real crawler got (Common Crawl)Common Crawl's index didn't answer us. It is slow at times; run the check again in a minute.Couldn't check
- Search and AI answers may quote the pageNo noindex, nosnippet or short max-snippet, in the page or its headers, so Google can show it and AI Overviews and AI Mode can quote it. 2 passages are marked data-nosnippet: that text stays out of snippets and AI answers, which is fine if you chose it.Good
AI crawlers can read this page. 2 things would make it clearer to them.
9 good · 2 could be better · 0 problems · 1 couldn't check
What to fix, in order
- Ready to be quoted in an AI answer. Give each question your buyers ask its own page: the question as a heading, the direct answer in the first lines, a list or table, your sources, and a named author. One article per question does it. an article
- www and http:// lead to your page. Nothing to change unless third-party SEO tools matter to you. If they do, your host or CDN can lower its bot-challenge level. That is worth weighing, because the same filter also guards pages that are never cached, such as checkout and cart. about 10 min Quick win
A page from your sitemap, beside this one
We opened /seo-article-writing/, the first content page in your sitemap, and GPTBot got no page (HTTP 429), so there is nothing to compare.
Technical details: what each check read
- 1. robots.txt for the index-class bots open to every index-class botGPTBot: open · its own group
OAI-SearchBot: open · its own group
PerplexityBot: open · its own group
ClaudeBot: open · its own group
Claude-SearchBot: open · the * group
Googlebot: open · the * group - 2. Server HTML against a browser full server HTMLGPTBot got 5562 words (HTTP 200) · a browser got 5562 words (HTTP 200)
- 3. Index class and user class, side by side both classes read the pageindex: GPTBot 5562 words (HTTP 200) · user: ChatGPT-User 5562 words (HTTP 200) · control, an invented bot: 5562 words (HTTP 200)
- 4. Title, meta description, H1 title · description · 1 H1« Video Editing and SEO Articles in 72 Hours | IntentClose » · « Short-form video, YouTube edits and SEO articles buyers actually act on. Drafted by AI, finished by a human, post-ready in 72 hours from $20. 1000+ deliveries. »
- 5. /llms.txt present
- 6. JSON-LD types Answer, Brand, DefinedRegion, FAQPage, ImageObject, MerchantReturnPolicy, MonetaryAmount, Offer, OfferShippingDetails, OnlineBusiness, Organization, Person, PostalAddress, Product, QuantitativeValue, Question, SearchAction, Service, ShippingDeliveryTime, UnitPriceSpecification, VideoObject, WebPage, WebSite
- 7. Sitemap present · declared in robots.txthttps://intentclose.com/sitemap_index.xml
- 8. dateModified or a visible « updated » line 2026-10-15T10:27:30+00:00 (JSON-LD dateModified)
- 9. Answer-ready: questions, a list or table, short paragraphs, sources, an author 2 of 5: list, sources1 question headings · 150 paragraphs of 5+ words · sources: cookiedatabase.org, fiverr.com
- 10. The www and http:// versions http:// 301 to https://intentclose.com/ · www.intentclose.com (browser) 301 to https://intentclose.com/ · www.intentclose.com (script) 403 · intentclose.com (script) 403
- 11. Common Crawl: what its real crawler got Common Crawl's index didn't answer
- 12. Indexing and snippet directives nothing limits indexing or snippets · 2 data-nosnippet · no X-Robots-Tagmeta robots: follow, index, max-snippet:-1, max-video-preview:-1, max-image-preview:large · Source: https://developers.google.com/search/docs/appearance/ai-features
Checked 2026-10-16 17:47 UTC for https://intentclose.com/. One fetch per crawler class, no model asked.
Does ChatGPT name you? Ask it, and three more
Readable is the entry ticket. With a free account, we ask ChatGPT, Gemini, Claude and Perplexity the two questions a buyer asks about what you sell, and build your report:
- whether each AI names you, and whether it links to your site
- who it names instead, in a chart
- which sites the answers come from: articles, best-of lists, YouTube, directories, marketplaces
- what to do next, in order, from what they cite
- what ChatGPT says your business is, beside your own description
Not the same as asking your own ChatGPT. Yours remembers you: your chats, your memory, your location. The buyer questions go out anonymously, with no account history, from a fixed market, and without your brand name in them. You see what a stranger shopping for what you sell sees.
One report a week, free, no card. You read every answer in full, so you can check our work.
A real report: IntentClose, 2026-10-07
The other names, the sites the answers cite and what to do next are in the full report.
Then give the AIs something to quote
Articles · IntentClose
AI answers are built from pages that already answer the buyer's question. In the four AI panels we run every day, 43% of the 951 pages they cited on 2026-10-07 were articles, guides, best-of lists and comparisons, and 62% of answers cited at least one.
I write the pages AI answers draw on: 1,750+ word articles built on the questions your buyers ask, with images, an FAQ and internal links. A human proofreads every one, and each is back in 72 hours, in English or French.
- $30 one article to try
- $25 an article in a 4-pack
- $20 an article in a 30-pack
Your free report names the four articles I would write first for your market. Create a free account.
I run IntentClose, so I'm biased. The twelve checks above read the same whoever writes your pages.
What the check reads, and why it matters
Can ChatGPT read my website?
It reads what your server sends. GPTBot, ClaudeBot and PerplexityBot fetch the raw HTML and don't run JavaScript, so text that appears only after scripts run is invisible to them. Check 2 compares what GPTBot receives with what a browser shows.
Does blocking GPTBot remove me from ChatGPT?
No. GPTBot collects pages for OpenAI's model training. ChatGPT search reads through OAI-SearchBot, and ChatGPT-User opens a page when someone asks about it. Blocking OAI-SearchBot is what keeps a site out of ChatGPT search answers. Anthropic splits the same way (ClaudeBot for training, Claude-SearchBot for search), and PerplexityBot is Perplexity's search crawler.
Is llms.txt worth adding?
It's optional. llms.txt is a proposed standard: a plain-text file that tells AI tools what your site offers. It takes about ten minutes, so we list it, and we never count its absence as a problem. A file that is there but unusable (no « # » title line, no links, or almost empty) reads « could be better »: Chrome's Lighthouse fails it, so a broken file costs more than none.
My page passes every check. Why don't the AIs mention me?
Passing means the AIs can use the page. When a buyer asks a question, they pick from the pages that answer it best, and most of those sit on other sites: in the AI panels we run, 43% of the 951 pages cited on 2026-10-07 were articles, guides, best-of lists and comparisons. The free account shows which pages they pick in your market.
What does the free account add?
Once a week, it asks ChatGPT, Gemini, Claude and Perplexity two questions a buyer asks about what you sell, then shows every answer in full: whether you are named, who is named instead, which sites the answers cite, and what to do next. A third question asks ChatGPT what your business is and who it is for, and sets its answer beside your own site's description.
Can I run more than one report a week?
Yes, with credits. Your free report stays free, once a week; an extra report costs 5 credits, and packs start at $5 for 50 credits. A daily report is an option too, about 150 credits a month, and it pauses by itself when credits run out.
What happens to the address I check?
We keep the address you check, its result and where on the site you started, never your IP address. We use them to improve the check and to write to businesses we think we can help. If you ask for the result by email, we keep that email to send it and may follow up once about it. You get the IntentClose letter only if you tick the box.
Why is there no score?
Because a single number hides what it read. A scanner that receives a bot-check or error page instead of your site still prints a number: on 2026-10-07 an SEO audit scored intentclose.com 46 out of 100 for a missing description, headings and structured data, all of which the page has. Its scanner had been handed the host's « checking your browser » page. Here every check shows what it read, with the HTTP status beside it, and a page we couldn't read says so instead of failing.
Isn't that the same as asking ChatGPT myself?
No. Your own ChatGPT or Gemini answers with your memory, your chat history and your location, so it can name you because you once told it about your business. Our questions go out anonymously: logged out or through the API, with no account history and no memory, from a fixed market (the United States or France), and your brand name is never in the two buyer questions, only used to read the answers (the third, « what is your business? », names you on purpose and is kept out of every count). You see what a stranger shopping for what you sell would get. The report also asks four AIs two wordings at the same moment, and pulls out what a chat window hides: every site the answers cite, sorted by type, and who gets named how often.