AI agents are no longer theoretical; they are integral to production infrastructure, making billions of daily HTTP requests. Previously, when these agents encountered errors, they were met with the same verbose, human-designed HTML pages meant for browsers. This created a significant inefficiency, providing agents with clues rather than clear instructions and wasting valuable tokens.
Addressing this gap, Cloudflare has begun returning RFC 9457-compliant structured error responses in both Markdown and JSON formats. This replaces the cumbersome HTML with machine-readable instructions, offering agents actionable guidance instead of obstacles.
From Clues to Instructions
Instead of a generic 'You were blocked' message, an AI agent will now receive specific instructions like 'You were rate-limited, wait 30 seconds and retry with exponential backoff.' Similarly, an 'Access denied' error will be replaced by 'This block is intentional: do not retry, contact the site owner.'
These structured responses dramatically improve efficiency, cutting payload size and token usage by over 98% compared to traditional HTML error pages. This saving compounds quickly for agents navigating complex workflows that encounter multiple errors.
Machine-Readable Errors by Default
Cloudflare's new system automatically delivers RFC 9457-compliant structured responses for all 1xxx-class errors, which cover platform-level issues like DNS resolution failures, access denials, and rate limits. This applies network-wide without any configuration needed from site owners, while browsers continue to receive the familiar HTML experience.
The daily brief
Get the free daily brief with the day's AI news, new AI startups and funding rounds every morning.
Join 64,000 other subscribers.
The structured responses include YAML frontmatter for machine-readable fields and prose sections for explicit guidance. For JSON responses, these fields are presented as a flat object. Key fields like error_code, retryable, and owner_action_required enable agents to classify failures, implement backoff logic, and determine when to escalate.
The JSON responses adhere to the RFC 9457 standard for HTTP API error reporting, ensuring compatibility with standard HTTP clients. Cloudflare extends this standard with operational fields such as error_category, retry_after, and ray_id, providing deterministic data for logging and support.
Size Reduction and Token Efficiency
The impact on efficiency is substantial. A comparison for a 'rate-limit' error (1015) shows a Markdown response uses 58.5x less data and 64.5x fewer tokens than its HTML counterpart. JSON responses offer similar gains, with an 48.1x reduction in size and 55.7x fewer tokens.
This reduction in size directly translates to lower token costs for AI agents, which is critical for managing model spend and improving recovery loops when multiple errors occur.
Clear Actions for Every Error
Each 1xxx error is mapped to a specific error_category, transforming error handling into straightforward routing logic. Categories range from access_denied and rate_limit to dns and config errors, each with clear directives for agent action.
The retryable and owner_action_required fields provide explicit control flow signals. This allows agents to move beyond brittle heuristics, like simply retrying on a 429 status code, to implementing precise logic based on stable, structured data.
Agents can now parse these structured fields once and branch on stable data. A common pattern involves checking if retryable is true, waiting for retry_after seconds, and retrying. If owner_action_required is true, the agent escalates; otherwise, it fails fast.
How to Utilize Structured Errors
To receive these structured responses, AI agents should send the Accept header with values like text/markdown, application/json, or application/problem+json. For testing, specific error codes can be accessed via the /cdn-cgi/error/ endpoint on any Cloudflare-proxied domain.
Cloudflare's implementation ensures that the first explicitly requested structured type takes precedence, providing a predictable response format. This shift fundamentally changes how AI agents interact with web infrastructure, moving from inference based on human-readable pages to direct execution of machine-defined policies.
