Deutsche Börse uses GenAI for data migration

Deutsche Börse Group is leveraging generative AI to tackle the monumental task of migrating thousands of analytics notebooks. Facing a 2027 deadline to move from Cloudera's soon-to-be-decommissioned Zeppelin notebooks to Databricks, the financial services giant developed a custom Databricks App to automate the process.

Visual TL;DR. Cloudera Deadline leads to Custom Databricks App. Thousands of Notebooks leads to Custom Databricks App. Custom Databricks App uses GenAI for Logic. Custom Databricks App includes Automate Structure. GenAI for Logic leads to 90% Time Cut. Automate Structure leads to 90% Time Cut. 90% Time Cut leads to Faster Migration.

Cloudera Deadline: 2027 deadline to move from soon-to-be-decommissioned Zeppelin notebooks
Thousands of Notebooks: over 2,000 users and a vast number of notebooks with complex logic
Custom Databricks App: developed a custom Databricks App to automate the migration process
GenAI for Logic: generative AI reconstructs complex notebook logic, not just structure
Automate Structure: app handles deterministic aspects: paragraph to cell, syntax translation
90% Time Cut: cutting redevelopment time by over 90% from manual rewrite
Faster Migration: enables migration from hours to minutes for thousands of notebooks

Visual TL;DRQuickExplainDeeper

The challenge involved over 2,000 users and a vast number of notebooks, many containing complex logic and institutional knowledge. A manual rewrite would have taken years, prompting the development of an AI-assisted solution. The core innovation, as detailed on the Databricks blog, lies in separating structural conversion from logic reconstruction.

Automating the Structure, AI for Logic

The Databricks App handles the deterministic aspects: transforming Zeppelin's paragraph format into Databricks cells, translating interpreter syntax, and reformatting metadata. This structural conversion preserves the original content precisely.

The true power comes in the next step: logic reconstruction. For each converted notebook, the app generates a context-aware prompt tailored to Deutsche Börse's specific Zeppelin environment, including custom interpreters and data source references. This prompt is fed into Databricks Genie, an AI assistant that asks clarifying questions and rebuilds the notebook's logic.

This hybrid approach, automating the predictable and delegating the complex to AI, is key to the success of this Databricks generative AI migration.

From Hours to Minutes

The result is a dramatic reduction in migration time. Notebook redevelopment that once took hours now takes a mere 15-20 minutes per notebook. This significantly accelerates a critical cloud transformation initiative.

Crucially, the tool democratizes the migration process, requiring less deep technical expertise from business users. The Databricks Streamlines Lakehouse Migrations initiative is exemplified by this project, showcasing how platform capabilities can be combined with AI for efficiency.

Lessons Learned

Deutsche Börse highlighted several key takeaways: avoiding over-engineering, recognizing the limitations of rule-based systems for heterogeneous content, and the critical importance of context in AI prompts. Early collaboration with the platform team was also vital.

This project demonstrates that AI-assisted migration is not a future concept but a present reality, transforming daunting cloud migration challenges into manageable, scalable workflows.

© 2026 StartupHub.ai. All rights reserved. Do not enter, scrape, copy, reproduce, or republish this article in whole or in part. Use as input to AI training, fine-tuning, retrieval-augmented generation, or any machine-learning system is prohibited without written license. Substantially-similar derivative works will be pursued to the fullest extent of applicable copyright, database, and computer-misuse laws. See our terms.