Databricks AI Gateway Tames Tool Access

Databricks is rolling out its Databricks AI Gateway, a new feature designed to streamline how AI agents interact with external tools and data sources. The core challenge addressed is the complex authentication process required to connect agents to services like GitHub, Glean, and Atlassian.

Traditionally, integrating AI agents with these external Model Context Protocol (MCP) servers has been a bottleneck. Each service demands its own OAuth app registration, client secrets, and token management, often leading to weeks of development time for what should be a straightforward connection.

Unified Governance for External Tools

The Databricks AI Gateway integrates directly with Unity Catalog, treating external MCP servers as first-class citizens. This allows administrators to register and govern these connections just like any other data asset. Fine-grained permissions can be applied, and all interactions are logged for comprehensive auditability and traceability.

This unified approach means teams can discover and deploy MCP servers from partners directly through the Databricks Marketplace.

Secure, User-Centric Access

A key security feature is the ability for agents to act on behalf of the end-user. This ensures that an agent accessing, for example, User A's email or private GitHub repositories, only has visibility and permissions aligned with User A's specific access rights. This prevents the need for overprivileged service accounts.

Administrators can further refine agent capabilities by scoping OAuth permissions per connection. For instance, a GitHub connection could be limited to read-only repository access, enhancing security. This approach to Managed OAuth for AI agents is crucial for enterprise adoption.

Simplifying Authentication Workflows

The AI Gateway abstracts away the complexities of OAuth flows. Users can select a provider from a dropdown, and Databricks manages the entire authentication lifecycle server-side. This eliminates the need for manual OAuth app registration and secret management for each service.

Initial supported providers include Glean, GitHub, Atlassian (Jira and Confluence), Google Drive, and SharePoint, with more planned. This functionality works seamlessly across AWS, Azure, and GCP deployments of Databricks.

From Connection to Deployment

The process is designed for speed. Users create an external MCP server connection, choosing between per-user OAuth (recommended) or a shared principal. Databricks handles the backend authentication setup.

Connections can be validated directly within the AI Playground or programmatically using the DatabricksMCPClient. Once validated, agents can be deployed using Agent Bricks.

Post-deployment, MLflow Tracing provides end-to-end observability of agent interactions, including tool calls and MCP server exchanges. This complements Unity Catalog audit logs for a complete picture of access and activity.

The introduction of Managed OAuth for AI agents is set to accelerate the development and deployment of context-aware agents capable of acting on both internal and external data.