Databricks adds AI guardrails

Databricks is rolling out a new set of controls for its AI applications, aiming to prevent security breaches and ensure compliance. The company announced the beta release of Unity AI Gateway Guardrails, a feature designed to provide flexible oversight of AI model and agent behavior.

The move comes as organizations grapple with the risks associated with AI deployment. Databricks highlights that governing AI usage is complex, citing its own AI Security Framework which lists dozens of risks and controls. LLM guardrails are presented as a fundamental governance and security layer.

These guardrails serve multiple purposes: protecting sensitive company data from being exposed to models, preventing AI from generating harmful or offensive content, and ensuring that AI-generated text aligns with brand guidelines. They can also keep conversational AI focused on specific topics.

Databricks is rolling out a new set of controls for its AI applications, aiming to prevent security breaches and ensure compliance. The company announced the beta release of Unity AI Gateway Guardrails, a feature designed to provide flexible oversight of AI model and agent behavior.

Governing Generative AI for Marketing

Consider Acme Co., a fictional marketing firm using an AI assistant for campaign drafting. Their CIO mandates strict policies: no customer Personally Identifiable Information (PII) in prompts, screening for jailbreaks and prompt injection, and prohibiting harmful content generation.

Additionally, Acme wants to avoid disparaging competitors in campaign materials. To achieve this, the AI platform team configures a Unity AI Gateway Endpoint.

Building a Governed AI Endpoint

The team selects a general-purpose model and sets up inference tables for monitoring. They map business requirements to specific guardrail types:

• PII Detection & Redaction: Sanitizes input to prevent PII leakage.
• Jailbreak & Prompt Injection: Blocks input attempts to manipulate the AI.
• Unsafe Content Blocking: Blocks harmful or unsafe outputs.
• Custom Block: A tailored guardrail to prevent naming or disparaging competitors.

Setting up built-in guardrails involves selecting the type, configuring actions like redaction or blocking, and optionally adjusting the evaluator model for performance or cost. A log mode is available for testing new guardrails on live traffic without disruption.

Custom guardrails require more detailed prompts, specifying the business context, competitor names, and providing few-shot examples. The effectiveness of these custom guardrails can depend on the chosen evaluator model, with Databricks suggesting iterative refinement based on performance and cost.

Testing and Auditing Guardrails

The Acme team tests the endpoint with various prompts, observing guardrail behavior. Prompts containing PII are sanitized, while attempts at jailbreaking or generating disparaging content are blocked.

The testing revealed that custom guardrails needed refinement. Iterations on the prompt, along with switching to a more capable evaluator model like GPT-5.4-mini, improved reliability. Databricks recommends capturing live traffic data to further tune custom guardrails for precision, recall, cost, and latency.

Guardrail activity is logged in inference tables, providing detailed insights into request status, token usage, and evaluator responses. These tables allow for tracing guardrail decisions back to client calls, enabling the creation of reports and dashboards for usage analysis and troubleshooting.

This granular visibility helps in validating user sessions if guardrails prove overly sensitive.

LLM Guardrails in Unity AI Gateway are now in beta, encouraging users to implement them for endpoints handling sensitive data or customer-facing outputs.