# Do AI Readiness Checkers Test What AI Bots See _Most GEO scores check presence, not access. This explains how to test what AI bots actually receive and use._ **Published:** 2026-11-10 **Source:** https://www.startuphub.ai/blog/do-ai-readiness-checkers-test-what-ai-bots-see --- Most AI readiness checkers test for presence, not access. They will tell you schema exists, llms.txt exists, and robots.txt does not block AI bots, without checking whether an AI agent can actually read the page. A useful test looks at what the bot receives and can use, starting with raw markup, specific bot permissions, and conflicting directives. ## Why presence checks fall short A green check for schema can hide an empty or generic implementation that adds no useful facts for synthesis. The same applies to llms.txt. A file can exist at the expected path yet contain vague marketing copy, broken links, or pointers to pages that themselves require interaction to display prices, specs, or contact details. Presence tells you almost nothing about whether the facts an agent needs are reachable in plain markup. JavaScript is where many scores go wrong. A page can look complete in a browser while serving a thin shell in its initial markup, with product details, pricing, reviews, or documentation injected after rendering. Many AI bots and agent fetchers do limited or no script execution during discovery, so they never see that injected layer. If your key facts only appear after interaction, a tool that renders like Chrome will overstate readiness. ## What to test if you want what bots receive The practitioner version of this test is less glamorous than a score. You compare what different requesters get and you read permissions literally, bot by bot, rather than accepting a simple AI bots allowed label. - Raw markup versus rendered view for key templates, such as product, pricing, docs, and contact. View source and confirm headlines, core facts, and links are present without running scripts. - Robots rules by named bot, not as a group. Training bots, search bots, and user triggered fetchers often have separate tokens and separate policies. Allowing a training bot does not open access for search or user triggered fetchers, and site hosts sometimes block AI categories by default. - Conflicts between robots and indexing directives. A page that is disallowed cannot be fetched, so a noindex tag on that page will never be seen. The result is the opposite of what many owners intend, with the page remaining unvisited rather than cleanly excluded. - Edge controls that sit outside robots. Bot management in a content delivery network can block verified AI bots by identity even when robots allows them. A checker that sends a matching user agent string from its own server will not trigger that edge decision. - Alternate representations served by user agent. Returning markdown only to AI user agents at the same address can read as cloaking, even when the intent is helpful. Safer patterns use the same content for all requesters or a clearly separate representation with consistent links. ## Where tools mislead, even with good intent Markdown copies are a frequent source of false confidence. Some checkers reward a site for offering a markdown variant, while search practitioners in the thread called that effort wasted if the underlying facts are thin or duplicated. A markdown file does not fix missing structured data, vague pages, or facts trapped behind interaction. It can also create maintenance drift, where the markdown copy goes stale while the visible page changes. Edge blocking is the harder blind spot. Most checkers fetch with a familiar bot label but they do not arrive from verified bot address ranges, so network level bot rules treat them as ordinary visitors. That means a report can say access is fine while a genuine AI bot is challenged or refused before it ever reads robots. Community feedback converged on the same fix. Treat the checker as a markup and policy review, then confirm with server logs filtered to verified bot identities to see what was actually allowed through. Of the 24,505 startup sites StartupHub.ai has scanned for Agent Readiness, 2 percent score 70 or above, and the same scan looks at a live site the way an AI agent reads it, covering llms.txt, MCP, schema, clean markup, and whether key facts are reachable without JavaScript, returning a 0 to 100 score with a fix list. ## When deep AI checks matter less If your pages are server rendered, use stable addresses, carry specific schema tied to visible facts, and have no separate bot rules at the edge, most AI readiness work collapses back into traditional search hygiene. Clear titles, concise introductions that state what the page offers, discoverable links, fast templates, and accurate structured data carry over directly into retrieval and synthesis. Several replies made this point bluntly. Old style optimisation remains the foundation for being cited, because retrieval still depends on finding and understanding pages before any model writes an answer. Rely less on specialised scores when your traffic and citations already come from a small set of evergreen pages. In that case, hand review those templates, fix script dependent facts, align robots with index directives, and clarify llms.txt so it points to maintained sources. Revisit edge bot settings only if logs show genuine AI demand being refused. ## Common questions **Should I serve markdown just for AI bots?** Usually no. Keep an accurate representation for all requesters and make the facts available in plain markup. If you publish a markdown or text variant, keep it consistent with the visible page and link it openly rather than swapping content by user agent. **Why does a tool say GPTBot is allowed while my logs show it blocked?** Robots and edge controls are separate layers. Robots states a preference, while bot management can enforce identity based blocking before a fetch happens. Check named bot rules, host defaults, and edge settings, then verify with logs for verified bot traffic. **Can a readiness score hurt if I follow it literally?** Yes, when it rewards presence over correctness. Adding generic schema, a thin llms.txt, or a user agent specific markdown copy can raise a score while leaving facts unreadable or creating policy conflicts. Fix readability and permissions first, then use the score to track gaps. [Scan your site for AI readiness](https://www.startuphub.ai/agent-readiness) Free, no account needed. ## Questions people ask ### Should I serve markdown just for AI bots? Usually no. Keep an accurate representation for all requesters with facts in plain markup, and link any text variant openly rather than swapping by user agent. ### Why does a tool say GPTBot is allowed while my logs show it blocked? Robots and edge bot management are separate. A checker sees the robots preference, while the edge can block verified bot identities before any fetch. ### Can a readiness score hurt if I follow it literally? Yes, when it rewards presence over correctness. Generic schema or a thin llms.txt can raise a score while key facts stay unreadable. ### The thread that asked Quoted as posted. The answer above is ours. > I’ve been looking at tools that score a site’s “AI search readiness”. Most of them seem to check the same things: schema present, llms.txt present, AI bots not blocked in robots.txt. Presence, not correctness. What I haven’t seen yet is a checker that tests things like: raw HTML vs rendered, since many AI crawlers don’t run JavaScript robots.txt rules per bot type (training vs search vs user-triggered fetchers), not just “AI bots allowed: yes” a page that’s disallowed in robots.txt while also carrying noindex, so the noindex is never seen markdown copies served at the same URL depending on... > > r/TechSEO, u/sashaverg, Oct 14, 2026. [View on Reddit](https://www.reddit.com/r/TechSEO/comments/1x5milt/do_any_geo_aireadiness_checkers_test_what_ai/) --- Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory. © StartupHub.ai. All rights reserved. You may not republish this article in full without a license. Search engines and AI research tools may crawl and summarize for reference. Bulk reproduction or model training on this content requires a license. See https://www.startuphub.ai/terms.