Jure Leskovec on Relational Foundation Models

In a recent AI Podcast episode, Jure Leskovec, co-founder and chief scientist at Kumo, and professor at Stanford University, discussed the transformative potential of Relational Foundation Models for enterprise data. These models represent a significant advancement in applying deep learning to structured, relational data, moving beyond the typical unstructured text or image domains that have dominated recent AI breakthroughs.

Who Is Jure Leskovec?

Jure Leskovec is a distinguished researcher in the field of machine learning and artificial intelligence. His work at Stanford University and as a co-founder of Kumo focuses on developing novel AI models and applying them to complex, real-world problems. Leskovec is particularly known for his contributions to graph neural networks, recommender systems, and the analysis of large-scale data, including social networks and, more recently, enterprise data.

The full discussion can be found on TWIML's YouTube channel.

Relational Foundation Models for Enterprise Data

Leskovec introduced Relational Foundation Models as a new class of models designed to understand and reason over the inherently structured and relational nature of enterprise data. Unlike traditional machine learning models that might require extensive feature engineering or task-specific training, these foundation models aim to learn general representations of entities and their relationships directly from raw data. This approach allows them to be applied to a wide array of downstream tasks without significant adaptation.

The core idea behind these models is to treat enterprise data as a massive, interconnected graph. Entities, such as customers, products, or transactions, are represented as nodes, and the relationships between them, like purchases, interactions, or dependencies, are represented as edges. Leskovec explained that the models are trained using a self-supervised learning objective, akin to masked language modeling in natural language processing. Specifically, the models learn to predict masked entities or relationships within the data graph, allowing them to capture the underlying structure and semantics of the enterprise data.

Key Capabilities and Applications

Leskovec highlighted several key capabilities of Relational Foundation Models:

• Understanding Complex Relationships: The models can capture intricate, multi-hop relationships within the data, which are crucial for understanding complex business processes and customer behaviors.
• Generalizability: By learning general representations, these models can be fine-tuned for various downstream tasks, such as fraud detection, customer churn prediction, recommendation systems, and even scientific discovery in fields like drug development.
• Scalability: While challenging, the research aims to scale these models to handle the vast quantities of relational data present in large enterprises.

He elaborated on how these models can be applied to real-world scenarios, such as identifying fraudulent transactions by understanding complex webs of suspicious relationships between entities, or predicting customer behavior by analyzing their interactions and relationships with products and services.

The Road Ahead

Leskovec emphasized that while the potential is immense, scaling these models to the complexity and volume of enterprise data remains a significant research and engineering challenge. However, the ability of Relational Foundation Models to learn from raw, structured data and generalize across diverse tasks represents a promising direction for unlocking the value hidden within enterprise information.