Databricks Unlocks Database Evolution

Twenty years after its initial conception, the principles of evolutionary database development are finally becoming operationally viable at scale. A key constraint has always been the difficulty of managing shared database resources, but Databricks is changing that with its Databricks Lakebase.

StartupHub data

Companies working on this

Profiles of the companies named in this story, with funding and a one-liner from our database.

Databricks

$190.0B

A unified data analytics and AI platform built on the lakehouse architecture.

This platform introduces a radical approach to database branching, leveraging copy-on-write technology. This allows for the creation of a one-second, zero-storage-at-creation branch of even terabyte-scale production databases. This capability directly addresses the limitations that previously made practices like "everybody gets their own database instance" aspirational.

The End of Aspirational Practices

The original seven practices for evolutionary database design, outlined in 2003, faced significant hurdles. Practice #1 (DBA collaboration) was hampered by synchronous, gating reviews due to the high cost of schema changes. Practice #4 (everyone gets their own database) was often infeasible due to licensing and infrastructure costs, leading teams to settle for shared, contention-filled development databases.

Furthermore, Practice #5 (continuous integration) struggled with isolation, as pipelines ran against shared targets. Practice #6 (all changes are refactorings) lacked adequate practice spaces, and Practice #7 (developers update on demand) was limited by the inability to experiment safely without impacting others.

Lakebase: The Technology Shift

Databricks Lakebase, a managed Postgres database built on the lakehouse architecture, decouples compute from storage. Its data resides on durable object storage, while the Postgres engine scales independently. This architecture makes copy-on-write database branching practical at production scale.

Branching becomes a rapid metadata operation, creating a new pointer to shared storage. Only modified pages diverge, preserving the parent. This zero-copy approach, completing in seconds regardless of database size, removes the technical cost barrier.

The result? Per-developer, per-pull request, and per-experiment branches are now routine. Mocks and in-memory database substitutes in testing become obsolete, replaced by real Postgres environments.

Emerging Practices for 2026

With Lakebase's capabilities, the original practices are now fully realized, and new ones emerge. DBAs can review schema diffs asynchronously on pull requests, shifting their focus from gatekeeping to design collaboration.

All database artifacts, including schema diffs and migration test results, are version-controlled alongside application code. Database changes are now mandated migrations, with idempotency as a new authorship rule for automatic deployment.

Every developer gets their own database instance, operational at granular levels. Continuous integration is enhanced, with every pull request running on its own isolated branch.

All database changes are refactorings, with cheap rehearsal spaces available to test migrations at production-like sizes. Developers can update their databases on demand in one second, with isolated, production-shaped data. Destructive testing becomes a default option with zero blast radius.

Automating the Workflow

The real power lies in automating these practices within CI/CD pipelines. GitHub Actions workflows can automatically create per-PR branches, run migrations and test suites against real Postgres, and post schema diffs as PR comments for asynchronous DBA review.

Branch cleanup on merge ensures ephemeral pipelines. These workflows enforce that every PR gets its own database, making branches transient pipeline properties rather than developer burdens.

The core concept of copy-on-write database branching fundamentally transforms the development lifecycle.