Imperial College London Boosts Dementia Care with Data Platform

Imperial College London is accelerating dementia research by overhauling its data infrastructure. The institution's UK Dementia Research Institute Centre for Care Research and Technology (CR&T) has implemented a modern data platform to unify disparate data sources, aiming to provide faster, more reliable insights for both care providers and researchers. This initiative leverages advanced analytics to improve outcomes for individuals living with dementia.

For many with dementia, communicating symptoms like pain or fever is difficult, complicating diagnosis and delaying treatment. Subtle health shifts, such as changes in sleep or movement, can be critical indicators. CR&T's Minder system continuously tracks these signals using in-home sensors, sleep monitors, and electronic health records to build a real-time health profile. This data helps detect infections early, reduce hospitalizations, and support longer independent living at home.

However, the existing platform struggled to scale with growing data volumes and device numbers, hindering timely insights. This led to three core challenges: competing workloads slowing innovation, tightly coupled storage and compute driving up costs, and limited data access for researchers and non-technical stakeholders like clinicians. These issues impeded the translation of research findings into clinical practice.

Building a Scalable Research and Care Environment

CR&T re-architected its platform to separate operational and analytical workloads. IoT data is now ingested and validated via a Kubernetes layer before being stored in Delta Lake on Azure Data Lake Storage, following a medallion architecture (bronze, silver, gold). This modular approach ensures scalability without impacting operational systems. Electronic Health Record (EHR) systems remain optimized for interoperability using the FHIR standard, facilitating seamless data exchange with the NHS. Early deployments are embedding remote monitoring data directly into clinical workflows.

Centralized governance was implemented using Unity Catalog, enabling fine-grained access control for diverse research teams and collaborators. Databricks now serves as the dedicated analytics layer, providing a unified environment for data exploration, model building, and collaboration, independent of production workflows. The team is also evaluating MLflow for streamlining model deployment and maintenance, building upon their existing use of Kubeflow.

This approach to data governance in healthcare research is crucial for managing sensitive patient information.

Accelerating Research Velocity and Accessibility

Unity Catalog's granular access controls span multiple dimensions, including study partners, user roles, and data sensitivity, allowing for effective permission management. The platform also streamlines the research-to-production workflow, accelerating insight development and sharing. Analytical pipelines are standardized and made reusable, reducing duplicated effort and providing researchers with robust templates for new datasets. This focus on healthcare data platform modernization is key to operational efficiency.

Accessibility for clinicians and other non-technical users has significantly improved. Databricks dashboards now present IoT device health, behavioral trends, and cohort-level insights intuitively. Integrations within existing monitoring systems are being tested, allowing clinicians to access insights directly within their familiar tools. The platform also ensures reproducibility by storing every data point with its original timestamp, allowing historical reconstruction of data views.

The CR&T’s experience underscores a broader trend in healthcare: the future of care relies on transforming fragmented data into actionable insights. As organizations increasingly adopt connected devices and AI analytics, the challenge shifts from data collection to building accessible, trustworthy, and usable systems for decision-makers. This healthcare data platform modernization effort highlights how a shared, trusted data platform can accelerate research, improve clinical decision-making, and ultimately enhance patient outcomes.

Tangible Gains in Productivity and Care

The new platform, built alongside existing systems, has yielded significant improvements. These include 100% uptime during migration, integration of new IoT data sources in as little as one month (down from six), and model development cycles reduced to approximately one month. The platform has seen rapid data growth, with millions of IoT data points ingested and a 50% month-over-month increase in adoption among non-technical users.

These metrics translate into more accessible, higher-value insights for people living with dementia. Ethan de Villiers, Data Engineer at CR&T, noted, “We’ve restructured how we work and made data more accessible. The Databricks analytical platform has already made clinical insights available for 581 people living with dementia in the last 5 months.” The team also estimates saving hundreds of engineering hours compared to building similar infrastructure from scratch.

The work at CR&T is ongoing, focused on surfacing objective, continuous data for a population that often cannot advocate for itself. The ability to compress the time between research insight and clinical decision-making, and to provide care teams with actionable evidence, is central to their mission. The CR&T's journey demonstrates that the primary barrier to data-driven care is often not the data itself, but ensuring the right people can access, trust, and utilize it, regardless of technical expertise.