Databricks Tackles Agentic AI Risks

Databricks is rolling out significant updates to its AI Security Framework (DASF v3.0), specifically targeting the emerging security challenges posed by agentic AI. The expanded framework introduces 35 new technical security risks and six mitigation controls designed to help organizations deploy autonomous AI agents with greater confidence.

This evolution acknowledges that AI agents are moving beyond passive information retrieval to actively querying databases, calling external APIs, executing code, and coordinating with other agents. This shift from "saying things" to "doing things" introduces a new threat landscape.

Introducing Agentic AI to DASF

Agentic AI is now the 13th system component within the Databricks AI Security Framework. The update specifically addresses the unique vulnerabilities associated with agent memory, intricate planning processes, and the integration of external tools. This includes crucial security considerations for the Model Context Protocol (MCP), an emerging standard for connecting agents to enterprise systems.

The core of agentic AI involves a loop: a request is broken down, a tool is selected and executed, and the output is analyzed to determine the next step. This real-time decision-making about data access and tool invocation creates novel risks, particularly around 'Discovery and Traversal,' where an agent might access data paths or tool interfaces unintended for the user.

The "Lethal Trifecta" of Agentic Risk

Databricks highlights the heightened risk profile when three conditions converge: access to sensitive data, processing untrustworthy inputs, and the ability to change system state or communicate externally. This combination can transform an agent into a "confused deputy" via indirect prompt injection, executing authorized actions with malicious intent.

The new risks and controls are categorized across three sub-components: the Agent Core (memory and reasoning), MCP Server (tool interfaces), and MCP Client (connection layers). Risks include Memory Poisoning, Intent Breaking, Cascading Hallucination Attacks, Tool Poisoning, and Prompt Injection within tool descriptions.

The framework also addresses inter-agent dynamics, introducing risks like Agent Communication Poisoning and Rogue Agents in Multi-Agent Systems, which become more complex as agent ecosystems scale.

Defense-in-Depth for Autonomous Systems

To counter these threats, Databricks emphasizes defense-in-depth strategies. The new controls include implementing least privilege for tools, ensuring agents have only the necessary granular permissions for their immediate tasks. This is crucial for developing robust Agentic AI security controls.

Human-in-the-loop oversight is recommended for high-stakes actions, with a design that accounts for potential reviewer fatigue. Sandboxing and isolation are critical for agent-generated code, preventing unauthorized access to broader systems.

Furthermore, AI Gateways and Guardrails, including monitoring, safety filtering, and PII detection, are essential for both input and output validation. Observability of an agent's "thought process" through agentic tracing is also highlighted, enabling auditing and detection of compromised reasoning.

These updates aim to provide organizations with the necessary tools to deploy autonomous agents safely, maintaining governance, observability, and robust security.

The full details are available in the Databricks AI Security Framework Agentic AI Extension whitepaper and the updated compendium, which maps these new risks and controls to industry standards.