Data Quality: The AI Strategy

The guiding principle for high-quality AI is, unsurprisingly, high-quality data. This means organizations must prioritize fixing data at its transactional source before attempting to filter or refine it downstream. As Databricks emphasizes, if you want clean water in your intelligence layer, you must fix the pipes first.

NYU Langone Health, a major academic health system, has embraced this philosophy. By migrating to a unified data and AI platform and retiring legacy systems, the institution is laying the groundwork for advanced AI applications. Chief Digital and Information Officer Nader Mherabi highlighted the importance of this foundational work, noting that the true potential of AI hinges on reliable data.

Fixing Data Quality at the Source

Mherabi likens data quality to water flowing through pipes: clean water at the source eliminates the need for extensive, costly filtering later. This approach involves investing in common transactional platforms, like a single electronic health record and ERP system, to ensure data consistency and establish authoritative sources for patient, financial, and operational data.

This discipline of fixing data at the transactional level transforms the utility of the data layer. Years ago, scattered patient data without unified identifiers posed significant challenges. Now, by mastering systems and data sources, NYU Langone enables meaningful cross-walking of critical information, connecting patient care data to clinical trials and financial records. This capability is essential for a comprehensive understanding of patient journeys and is a direct outcome of effective fixing data at the source.

What Unified Data Actually Unlocks

In healthcare, where data accuracy is critical, a unified data foundation prevents departmental conflicts over metrics. This consistency builds trust, a vital component for agentic AI systems. Without unified data, metrics will inevitably misalign, undermining AI performance, which is directly dependent on data quality.

The ability to deliver real-time insights is paramount, especially in high-acuity environments like emergency rooms. Retrospective reporting is insufficient; real-time clinical decision support can actively prevent misdiagnoses. This requires architectural support for real-time data feeds, enabling models to operate on current information and provide just-in-time guidance to clinicians.

Unified Governance is a Strategic AI Imperative

Discoverability and trustworthiness of data at scale are enabled by robust data governance. Tools like Databricks Unity Catalog are crucial, but the strategy behind them, defining master data sources, assigning ownership, and controlling exposure, is fundamental.

A well-implemented catalog ensures users can find the right data without duplication, underpinning all subsequent data and AI initiatives. This focus on governance is a key aspect of a comprehensive Databricks AI strategy.

Building a Data-Literate Community

The value of a unified platform is realized only when it's widely adopted. NYU Langone actively evangelizes its platform's capabilities across the institution, aiming to become a learning health system. This requires extending platform use beyond IT to clinicians, analysts, and scientists, supported by literacy programs and training.

Real-Time Insight Where It Matters Most

In critical care settings, insights must be immediate. NYU Langone deploys models in the ER that flag potential critical conditions for clinicians, acting as a safety net to prevent missed diagnoses. These models require real-time data feeds to provide timely, actionable advice without replacing clinical judgment.

Three Layers of Data Analytics

Analytics strategy now encompasses basic visualization, conversational AI tools for deeper inquiry, and delivering insights in various formats tailored to user needs. The ability to interact with machines in natural language offers new avenues for exploration.

Leaders must balance long-term strategy with AI's rapid evolution. Accept unpredictability, partner with adaptable platforms, and focus relentlessly on value creation, whether in patient care, operational efficiency, or patient experience. Continuous education is key to navigating the evolving AI landscape.

The organization's commitment to a unified data platform for healthcare underscores the critical link between foundational data quality and the successful deployment of advanced AI capabilities.