Matt Carey on AI Agents and Cloudflare's API

Matt Carey, an AI Engineer at Cloudflare, recently delivered a compelling presentation at AI Engineer Europe, outlining the critical role of APIs in empowering AI agents. Carey's talk, titled "Every API is a Tool for Agents," explored the evolution of how AI agents interact with the external world, moving from tightly coupled, bundled tools to a more distributed, shared model.

Who is Matt Carey?

Matt Carey is an AI Engineer at Cloudflare, focusing on the intersection of AI agents and the company's extensive API infrastructure. His work involves enabling AI agents to effectively utilize the vast array of services Cloudflare offers, a task that requires deep understanding of both AI capabilities and the practicalities of API integration.

Giving Agents Hands: Tool/Function Calling

Carey began by illustrating the fundamental concept of tool/function calling for AI agents, a mechanism that allows them to interact with the outside world. He used a simple example of an agent needing to know the weather in London. This requires the agent to make a tool call to a weather API, receive the result, and then process that information. This process, he explained, has been a cornerstone of AI agent development for some time.

From Bundled to Shared APIs

Historically, AI agents often had their tools bundled directly within their own systems. This meant that for every specific task, like sending an email or checking a calendar, a dedicated integration was needed. Carey highlighted the limitations of this approach, particularly as the number of available tools and services grew exponentially. The more practical and scalable solution, he argued, is to shift towards a shared API model. This involves making APIs accessible to agents, allowing them to discover and utilize them as needed.

Before the advent of the Multi-Modal Communication Protocol (MCP), the approach was largely bundled. "Every app simply implemented the same integrations," Carey noted. This led to redundant work and limited the flexibility of agents. The introduction of MCP, however, signifies a move towards shared, discoverable APIs.

The Challenge of Scale and Context Windows

A significant challenge in providing agents with access to a vast number of APIs is the limitation of context windows in language models. Carey pointed out that Cloudflare's API surface is enormous, with potentially millions of endpoints. Trying to load all of this into an agent's context window is simply not feasible. He elaborated on this by stating, "We filled the context window. 2.3 million tokens in Cloudflare's Open API spec." This massive amount of information would overwhelm even the most advanced models.

This constraint necessitates a more intelligent approach to API access, leading to the concept of "progressive discovery." Instead of overwhelming the agent with all available tools at once, the system should provide tools progressively, based on the agent's immediate needs.

Progressive Discovery: The Solution

Carey outlined three key methods for achieving progressive discovery:

• Command Line Interface (CLI): Agents can be exposed to a CLI-like interface that allows them to discover and select tools dynamically.
• Tool Search: A search-based approach where agents can query for specific tools based on natural language descriptions.
• Code Mode: The ability for the AI model itself to generate the necessary code to interact with APIs, based on their specifications.

He demonstrated how these methods allow agents to access only the relevant tools for a given task, thereby managing the context window effectively.

The Cloudflare MCP in Action

Carey showcased Cloudflare's Multi-Cloud Platform (MCP) as a practical implementation of these principles. He demonstrated how agents can interact with Cloudflare's vast API ecosystem through a user-friendly interface. By allowing agents to list workers, query for specific services, and even generate code against API specifications, Cloudflare is enabling a powerful new paradigm for AI agent interaction.

He highlighted the security implications of running untrusted code from agents, emphasizing the need for robust isolation mechanisms. Cloudflare's approach involves providing programmable sandboxes where agent code can be executed safely, ensuring that even if malicious code is executed, it remains contained and does not compromise the broader infrastructure.

The Future: Simpler Servers and Smarter Agents

Carey concluded by projecting the future trajectory of AI agent development and API interaction. He anticipates that servers will become simpler, with MCP servers acting as thin, interoperable layers that expose capabilities handled by the protocol itself. Furthermore, he foresees a significant trend towards smaller, more lightweight frameworks and a greater reliance on TypeScript for its type safety and developer experience.

Ultimately, the goal is to make it easier for developers to build and deploy sophisticated AI agents that can seamlessly interact with a vast array of services, unlocking new possibilities for automation and problem-solving.