Free. No credits. REST + MCP.

Agent Readiness API

POST any URL. Get back a 0-100 AI-readiness score across 18 standards — robots.txt, llms.txt, MCP server card, /.well-known/api-catalog, OAuth discovery, x402, Content Signals, and 12 more. Free for everyone, MCP-ready for agents.

Scan a URL API Reference
curl
POST /api/agent-readiness/scan
curl -X POST https://www.startuphub.ai/api/agent-readiness/scan \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com" }'

How it works

One POST. 18 checks in parallel. Ready-to-paste fix prompts.

1

POST a URL

Send { url } to /api/agent-readiness/scan. No auth needed for the public scan; rate-limited 10 scans / 10 min / IP. Or call scan_agent_readiness via MCP.

2

We run 18 checks in parallel

robots.txt, llms.txt, MCP server card, /.well-known/api-catalog, OAuth discovery, x402 payment endpoint, Content Signals, markdown content negotiation, schema.org markup, and 9 more standards.

3

Get a 0-100 score + per-check breakdown

Response includes the overall score, every check's pass/fail/partial state, and a copy-paste fix prompt for each failure — drop into Claude / ChatGPT to remediate.

The 18 checks

Every standard the scanner runs against your URL, with weights driving the 0-100 score.

robots.txt

Standard crawler directives + AI-bot allowlists (GPTBot, Claude-User, Perplexity, etc.)

llms.txt

Anthropic-proposed standard for LLM-friendly site summaries.

llms-full.txt

Extended llms.txt with full content for ingestion crawlers.

MCP server card

/.well-known/mcp-server.json — does the site advertise an MCP endpoint?

API catalog

/.well-known/api-catalog — RFC 9727 machine-readable API directory.

OAuth discovery

/.well-known/oauth-protected-resource for agent-friendly auth.

x402 payment endpoint

L402 / x402 micropayment readiness for agent commerce.

Content Signals

Cloudflare Content Signals — train / search / ai-input policy directives.

Markdown content negotiation

Does the site serve text/markdown when the Accept header asks for it?

sitemap.xml

Discoverability for indexing crawlers.

OpenGraph + Twitter cards

Surface metadata for social agents.

Schema.org JSON-LD

Structured data — Organization, Product, FAQPage, etc.

Canonical URLs

Prevents duplicate-ingestion from query-string variants.

HTTPS + HSTS

Trust signal that AI agents check before transacting.

AI-bot rate limits

Detects 429s / Cloudflare blocks against known AI crawler IPs.

Cookie wall detection

Agents can't consent — pages behind cookie walls fail to index.

JavaScript-only content

Pages that need JS to render miss most AI ingestion crawlers.

Server-side rendering

Confirms first-byte HTML contains the visible content.

Response shape

Score + category breakdown + per-check status + copy-paste fix prompts.

{
  "url":   "https://example.com",
  "score": 38,
  "category_scores": {
    "discoverability": 60,
    "ai_metadata":     20,
    "agent_protocols": 15,
    "auth_payment":    50
  },
  "checks": [
    { "id": "llms_txt",  "status": "fail", "weight": 8 },
    { "id": "mcp_card",  "status": "fail", "weight": 10 },
    { "id": "robots_txt","status": "pass", "weight": 4 }
    /* ... 15 more ... */
  ],
  "fixes": [
    {
      "check": "llms_txt",
      "prompt": "Generate an llms.txt for example.com. Include..."
    }
    /* ... per failed check ... */
  ]
}

Built for these workflows

Real production patterns from teams using the scanner today.

Pre-launch audit

Run before shipping a marketing site. Catch missing llms.txt, broken MCP cards, and AI-bot blocks before traffic ramps. Free unlimited self-checks via MCP.

Outbound prospecting

Score 50 prospect sites in a batch. The lowest scores are your best leads — they need help. Pair with discover_email to send personalised outreach.

Agency dashboards

Embed the score on a client portal as a real-time AI-readiness KPI. Track improvement month-over-month as standards evolve.

Agent self-checks

In an agent loop building or hosting a site, periodically scan the site's own readiness. Ship a report with the score + remediation prompts.

Score your site now

Free. No signup. No credits charged.

Run a free scan Read the docs

Frequently asked questions

What is the AI Readiness API?
A free public API that scores any URL on a 0-100 scale across 18 AI-agent-readiness standards: robots.txt, llms.txt, MCP server card, /.well-known/api-catalog, OAuth discovery, x402 payment, Content Signals, markdown content negotiation, schema.org markup, and more. Returns the overall score, per-check pass/fail/partial state, and copy-paste fix prompts for each failure.
Is it really free?
Yes. The agent-readiness scanner does not consume API credits. Anonymous (no auth) callers get 10 scans per 10 minutes per IP. Authenticated callers get effectively unlimited (the credit cost is zero).
How long does a scan take?
Median 5-10 seconds. We fetch the site, parse it, run 18 checks in parallel, and return the report. P95 under 30 seconds for slow origins.
How do I call it programmatically?
POST to https://www.startuphub.ai/api/agent-readiness/scan with { "url": "https://example.com" }. No Authorization header required. The response is JSON: { score, checks: [...], fixes: [...] }. CORS-enabled for browser use.
How do I use it inside Claude or Cursor?
Drop our MCP server URL (https://www.startuphub.ai/api/v1/mcp) into your client. The scan_agent_readiness tool becomes available — your agent can score URLs in any chat. Free even without an API key on the MCP path.
What are the 18 standards you check?
robots.txt (with AI-bot allowlist parse), llms.txt, llms-full.txt, MCP server card, /.well-known/api-catalog (RFC 9727), OAuth discovery, x402 / L402 payment endpoints, Cloudflare Content Signals, markdown content negotiation, sitemap.xml, OpenGraph + Twitter cards, Schema.org JSON-LD, canonical URLs, HTTPS + HSTS, AI-bot rate-limit detection, cookie-wall detection, JS-only content detection, and server-side rendering verification.
How is the score calculated?
Each check carries a weight. The score is a weighted sum normalised to 0-100. Foundational checks (robots.txt, sitemap.xml) carry less weight than cutting-edge ones (llms.txt, MCP card, x402). The full breakdown is returned in the response so you can see exactly where points were earned or lost.
What's the difference between the public API and the Clay integration?
Same scanner, two surfaces. The public API is free and rate-limited per IP. Our Clay integration (/integrations/clay) lets non-technical users drop the scanner into their Clay tables — 1 credit per scan, 1,500 free per month, then $0.005/scan. Use the public API for self-hosted agents; use Clay for sales-ops workflows.
Can I get a fix prompt I can paste into Claude / ChatGPT?
Yes — every failed check returns a remediation prompt designed for an LLM. Paste it into Claude, ChatGPT, or Cursor and the AI generates the file content (e.g., a draft llms.txt for your site, an MCP server card, a Content Signals header). Designed so non-technical site owners can fix issues in one chat turn.
How do I score a list of sites in bulk?
No dedicated bulk endpoint — call /api/agent-readiness/scan in a loop. With your free public-IP rate limit (10 / 10 min) you can score ~60/hr. With an API key the limit is effectively gone (zero credits charged). The MCP scan_agent_readiness tool also accepts a single URL per call; loop in your agent code.

Related

Free Scanner

No-code web tool. Score URLs one at a time.

Clay Integration

Drop a domain into Clay, get the readiness score.

Email Finder API

Pair with discover_email for outreach to low-scoring sites.

Full API Reference

Every endpoint and MCP tool documented.