Neo4j: AI on Lakehouse with Context in Shapes

Zach Blumenfeld, an AI Research Engineer at Neo4j, recently presented a workshop titled "AI on Your Lakehouse: Context Comes in Shapes, Not Queries." The session aimed to address the challenges of providing AI agents with the right context to accurately answer questions from both structured data warehouses and unstructured data lakes.

Blumenfeld explained that while accessing data using tools like text-to-SQL and vector search has become easier, a significant hurdle remains in how to effectively deliver relevant context to AI agents. Traditional methods often fall short when dealing with massive datasets or complex interdependencies, leading to agents that might be "confidently wrong." Neo4j's approach introduces an agnostic data model that represents data as a graph, providing structure for both documents and tables.

The "Shapes" of Data Context

The workshop introduced three specific "shapes" designed to imbue AI agents with better contextual understanding:

• Table of Contents: This shape, resembling a tree structure with links, is used to provide navigation for unstructured data like documents.
• Themes: Applied to unstructured data, this shape surfaces global patterns and groupings that might not be immediately apparent, helping to uncover unknown insights.
• Connections: This shape acts as a semantic layer on top of a data warehouse, establishing relationships between tables and enabling agents to understand how data elements connect.

Blumenfeld highlighted a use case involving a fictional auto repair chain, "AutoFix Group," which has a wealth of data including repair manuals, service bulletins, and a warehouse of repair logs. The goal is to build a co-pilot that can assist technicians on the floor by answering complex questions that require traversing this diverse data.

Overcoming Data Access Challenges

Blumenfeld elaborated on the limitations of existing AI approaches when dealing with large, disparate datasets. He noted that while vector search and text-to-SQL can retrieve information, they often struggle with providing the interconnected context needed for accurate answers, especially when dealing with complex queries involving multiple tables or proving negatives (e.g., identifying missing documentation). The core problem, he stated, is a context problem, not a query problem.

The Neo4j graph database, with its property graph model of nodes, relationships, and properties, is presented as a solution. By modeling data as interconnected entities, Neo4j's platform allows for easier traversal and understanding of complex relationships, which can then be exposed to AI agents through various tools and skills.

Hands-On Setup and Tools

The workshop provided a practical demonstration, guiding participants through setting up a codepace with pre-installed tools like Python, Neo4j, and Claude. Attendees were instructed on how to enroll in Graph Academy, configure their environment with API keys, and run scripts to build the "connections graph." This process involves extracting metadata from BigQuery to create a semantic layer, enabling the agent to query the data more intelligently.

Blumenfeld emphasized that the workshop focuses on using agents to help write Cypher queries, rather than manual coding, making the process more accessible and efficient for AI engineers. The session concluded with a look at the generated graph, showcasing the interconnectedness of the data and the potential for AI agents to leverage this structure for more insightful responses.