Databricks Unifies Data Governance

Databricks is blurring the lines between operational and analytical databases, bringing PostgreSQL workloads directly under the governance umbrella of its Unity Catalog. This move aims to simplify data management and security by consolidating disparate audit trails and access controls into a single plane.

StartupHub data

Companies working on this

Profiles of the companies named in this story, with founding year, headquarters, and a short description from our database.

Databricks

A unified data analytics and AI platform built on the lakehouse architecture.

Founded

2013

Location

San Francisco, United States

Valuation

$190.0B

PostgreSQL

A powerful, open-source object-relational database system known for reliability and extensibility.

Founded

1996

Location

Santa Barbara, USA

In its second part of the Lakebase series, Databricks detailed how its Databricks Lakebase can host production applications, offering rapid database branching and recovery. The key innovation lies in extending Unity Catalog's governance over these operational databases, a significant departure from traditional architectures where application databases and data lakes operate under separate security paradigms.

Unified Governance for All Data

Traditionally, auditing a simple table drop on a production database meant cross-referencing logs from multiple services like CloudTrail, pg_audit, and CloudWatch. This fractured approach becomes a compliance bottleneck.

With Lakebase embedded in Databricks and exposed via Lakehouse Federation, Unity Catalog can now directly manage access policies, audit trails, and lineage for operational data. This means standard Unity Catalog grants control data access, eliminating the need for complex, database-level role management.

Row-level security and data masking policies, already used for analytical data, can now be automatically applied to operational tables and their ephemeral branches.

A Single Pane for Auditing and Cost

Every action taken against an operational database within Databricks is now logged in Unity Catalog's system.access.audit table. This provides a unified audit trail, accessible via a single SQL query, eliminating the need to stitch together logs from disparate systems.

Furthermore, compute costs are automatically attributed by branch, project, and endpoint. This native integration with Unity Catalog's system billing tables offers granular cost attribution for ephemeral workloads, a feature often missed by traditional tagging strategies.

Shifting the DBA Role

This integrated approach automates routine tasks like environment provisioning and access grants, freeing up Database Administrators. DBAs can transition from managing a queue of operational tickets to focusing on strategic platform architecture, defining governance rules, and designing promotion workflows.

The audit trail and cost attribution data serve as a strategic dashboard, offering real-time insights into platform usage and guiding future investment.

Automated Governance for Branching Workflows

For development teams leveraging rapid branching, Unity Catalog's governance becomes critical. Attribute-level masking policies propagate automatically to new branches, preventing sensitive data exposure in non-production environments.

This automatic enforcement is crucial, as a significant percentage of data breaches occur in inadequately secured non-production environments. Governance must be automatic to scale with fast-paced development cycles.

New tooling like LakebaseOps handles routine platform tasks, while Lakebase MCP allows DBAs to define intent and manage access through AI agents and a dual-layer governance model.