Neo4j CEO on AI Agents and Ontology-based Data

Emil Eifrem, Founder and CEO of Neo4j, addressed the challenges of preparing data for AI agents at the AI Engineer World's Fair. He outlined a critical problem emerging in the last six to nine months: the manual, repetitive, and inefficient process of connecting AI agents to the vast and complex data sources within large organizations.

The Problem with 'Thick Agents'

Eifrem illustrated the issue using the example of an 'account opening agent' for a bank. Such agents typically consist of 'business logic' (interpreting intent, planning, acting) and 'data sources' (identifying, assessing, and resolving necessary data). While this setup works for individual agents, scaling it across an enterprise presents significant hurdles. In large organizations, data is fragmented across hundreds of databases, cloud storage, and data platforms. Each time a new agent is built, developers must manually locate, validate, and connect to these data sources. This process is not only time-consuming but also violates the 'Don't Repeat Yourself' (DRY) principle of software engineering, as changes to data sources require manual updates across multiple agents. Furthermore, this approach lacks a mechanism for learning or improvement, meaning agents do not become smarter over time.

Proposed Solution: The Ontology-based Semantic Layer

Neo4j proposes a solution centered around an 'ontology-based semantic layer' to address these challenges. Eifrem described this as needing 'thin agents on a smarter shared substrate.' This approach is built on three core pillars:

• Business-facing Ontology: This pillar defines the key concepts within an organization (e.g., customers, accounts, transactions) in a human-readable format, ensuring common understanding across teams.
• Technical Ontology: This layer captures the metadata of all data assets and sources within the enterprise, detailing their location, schemas, attributes, and physical locations.
• Execution Traces: This pillar involves collecting runtime signals from agent executions, including decisions, paths taken, outcomes, and errors. These traces provide data on what worked and what didn't, enabling learning.

Eifrem emphasized that these three pillars, when combined, solve the critical problems of data discovery, trust, 'don't repeat yourself,' and agent learning. The ontology provides a single, governed place to map business intent to data sources, informed by both human curation and observed execution performance.

Addressing the 'Markdown Files' Approach

While acknowledging that simple solutions like markdown files might seem appealing for defining agent logic, Eifrem cited insights from Swyx on the Latent Space podcast, stating, "Learn your databases. You can't vibe code with just markdown files." He indicated that these simpler methods are only part of the solution and not sufficient for complex enterprise needs.

Real-World Application and Future Directions

Neo4j has been implementing this solution with major organizations, including a Fortune 20 global bank, a large tech platform company, and a leading fintech company. Eifrem showcased how the ontology can represent a business process, like account opening, and map specific steps, such as 'Check Compliance,' to underlying data sources like 'Motor Vehicle Records' or 'Passport Verification' through the technical ontology.

Eifrem concluded by highlighting the broader application of graphs in AI, mentioning an upcoming 'Amazing Graph Track' at the conference featuring talks on various graph-based AI patterns. He also encouraged startups to explore the Neo4j startup program, which offers credits and dedicated solution engineering support.