Databricks' Contextual Policies Stop Slow Attacks

Cyber attackers are evolving tactics to bypass AI agent security, moving beyond simple prompt injection to more insidious "slow-burn" attacks. These methods break down a malicious goal into a series of seemingly legitimate actions, making them difficult to detect by traditional, single-step security checks. Databricks aims to counter this with its Omnigent platform and its implementation of contextual policies.

The Evolving Threat Landscape

Traditional security measures often evaluate each agent action independently. A slow-burn attack, however, relies on the cumulative effect of multiple, individually permissible actions. For instance, an agent might be instructed to read a document, then read another, summarize the content, and finally email it. Each step, viewed in isolation, appears to be standard operational procedure.

The danger becomes apparent only when observing the entire session. An attacker can exploit this by hiding malicious instructions within data fetched by the agent, a technique known as indirect prompt injection. This allows them to exfiltrate sensitive information by disguising data theft as routine tasks.

Contextual Policies as a Defense

Contextual policies in Omnigent address this threat by maintaining a memory of session activities. Unlike stateless policies that assess each action independently, these stateful policies consider the history of an agent's operations. This allows them to identify patterns of risky behavior that emerge over time.

In a demonstration, an agent tasked with vendor reviews was compromised via a tampered runbook. Without a contextual policy, the agent successfully exfiltrated confidential pricing terms by following the modified instructions, sending the summary to an external address. Each step was permitted individually.

Blocking the Attack in Real-Time

When a contextual policy was applied, the attack was thwarted. This specific policy tracked a session risk score, incrementing it with each sensitive document read. Once the score surpassed a predefined threshold, the outbound email function was blocked, preventing data exfiltration.

This demonstrates how contextual policies can identify and neutralize threats that individual action checks would miss. The policy didn't block the email itself, but rather the email action taken after a history of sensitive data access within the same session.

Tamper-Resistant Security

A critical feature of Omnigent's contextual policies is their resistance to tampering by the agent itself. Agents are provided with tools to browse and add policies, but lack any mechanism to remove, edit, or disable them.

Furthermore, adding new policies requires human approval, preventing an agent from quietly introducing weaker security rules. When multiple policies are in effect, any single denial takes precedence, creating a robust, layered defense that cannot be circumvented by a compromised agent.

The takeaway is clear: evaluating individual actions is insufficient against sophisticated threats. Contextual policies, by remembering session history and being enforced externally, offer a more resilient defense against slow-burn attacks.