NBCUniversal Cuts Costs With Databricks

NBCUniversal has slashed its data infrastructure costs by 30% following a strategic migration to the Databricks Data Intelligence Platform. The media giant moved from a slot-based reservation model to Databricks’ dedicated job compute, allowing parallel data pipelines to scale independently.

This shift means jobs now complete faster and consistently meet service level agreements. StartupHub.ai data shows Databricks holds a strong position with a score of 82/100, positioning it favorably against competitors like Snowflake (72/100) and Palantir Technologies (85/100).

The migration also unified analytics and machine learning capabilities on a single platform, bringing over 300 analysts onto the new system. This move supports NBCUniversal's need for agility during major content launches and live events without costly overprovisioning.

The Cost and Performance Challenge

As data volumes surged and analytics demands evolved, NBCUniversal's existing infrastructure struggled. A slot-based reservation system, while supporting parallel workloads, led to significant compute expenses and resource contention. This hindered the company's ability to make fast, data-informed decisions across marketing, product, and content teams.

Evolving cost considerations and the need for greater flexibility for advanced use cases like machine learning drove the search for an alternative platform. Enhanced governance around data lineage was also a key motivator.

Databricks Lakehouse Architecture as the Solution

NBCUniversal partnered with Databricks and the global consultancy EXL to migrate to the Databricks Data Intelligence Platform. This collaboration combined EXL's migration expertise with Databricks' data solutions to create a modernized infrastructure for data engineering, machine learning, and advanced analytics.

The Databricks platform offers a unified environment for diverse workloads, eliminating the need for multiple specialized tools. Its flexible architecture, built on Delta Lake and Apache Spark, handles structured and unstructured data efficiently. Native integration of MLflow supports complex analytics and machine learning, while granular compute control ensures each pipeline runs on dedicated resources.

A Phased, Partner-Led Migration

The migration involved transforming data structures, modifying code, and adjusting data formats. EXL and Databricks implemented a phased approach to minimize disruption.

Phase 1 involved a comprehensive discovery of existing workloads and codebases. Phase 2 focused on designing the target state architecture, including workspace structure, governance with Unity Catalog, and compute strategies.

Before a full rollout, a targeted MVP was executed using custom migration tools. EXL developed proprietary accelerators, including a SQL translation engine, an orchestration migrator, a data transfer orchestrator, and a validation framework. These tools automated processes that would have otherwise taken months of manual effort.

The migration proceeded in strategic waves, prioritizing high-value, lower-complexity pipelines. Each wave involved automated code generation, data synchronization, and rigorous quality assurance. This iterative process allowed for continuous refinement based on lessons learned.

The final phase involved running both the legacy platform and Databricks in parallel for extended validation. This ensured data parity, provided performance benchmarks, and built stakeholder confidence before the final cutover.

Tangible Business Impact

The migration resulted in a 30% reduction in data infrastructure costs. This was achieved by leveraging Databricks’ job-specific compute, which allows for automatic scaling up and down based on demand.

Operational flexibility improved significantly. The parallel execution model enhances agility, enabling NBCUniversal to respond effectively to business demands during high-traffic events. This capability is vital for the dynamic media and entertainment data analytics landscape, where timely insights can drive success.

The company also gained unified ML and analytics capabilities through MLflow and Lakeflow Jobs. This integration streamlines workflows for data scientists and analysts working with media and entertainment data analytics.