Teaching AI Agents to Master Spreadsheets

Nuno Campos, CTO & Co-Founder of Witan Labs, delivered a compelling presentation on "Teaching Coding Agents to do Spreadsheets" at AI Engineer Europe. Campos shared insights from Witan Labs' efforts to improve AI agent capabilities in handling spreadsheet data, a task that presents unique challenges compared to traditional code execution.

The Spreadsheet Challenge for AI Agents

Campos highlighted the fundamental difference in how humans and AI agents perceive spreadsheets. While humans can intuitively grasp tables, assumptions, and P&L summaries, Large Language Models (LLMs) initially process spreadsheets as a flat collection of cell values and formatting metadata. This lack of inherent structural understanding makes tasks like calculating revenue or identifying specific data points within a large spreadsheet a significant hurdle for AI agents.

The presentation detailed the journey of improving an AI agent's spreadsheet proficiency, which involved overcoming numerous challenges and "dead ends." Early attempts focused on breaking down the task into multiple specialized agents, each handling a specific function like block discovery, editing, or answering questions. However, Campos noted that these rigid architectures often proved insufficient. A key finding was that "rigid architectures don't win."

The REPL Solution and its Advantages

A significant breakthrough came with the adoption of a persistent Node.js REPL (Read-Eval-Print Loop) combined with a spreadsheet API. This approach fundamentally changed how the agent interacted with spreadsheets. Instead of making numerous discrete tool calls, the agent could now write and execute JavaScript code within the REPL. This allowed for "persistent state," meaning variables and workbook states could be maintained across calls, enabling the agent to build understanding incrementally.

Campos illustrated the difference with a 'Before vs. After' code example. Previously, an agent might require 10-15 tool calls to extract basic information, such as listing sheets, reading a range, finding cells, and reading a specific cell's value. With the REPL approach, these operations could be consolidated into a single, more coherent code execution. This not only simplified the agent's workflow but also improved its ability to reason about the interdependencies between different parts of the spreadsheet.

Lessons Learned and Generalizations

The Witan Labs team iterated on their approach, changing tools four times over four months, yet found that "domain knowledge improved results on every one of them." This underscored the critical role of understanding the specific domain, in this case, financial analysis, in guiding the AI's interactions with the spreadsheet.

The presentation outlined several key takeaways that generalize beyond spreadsheet tasks:

• Many small sequential tool calls are less effective than a single, well-crafted scripting language command.
• Building verification engines, including formula calculation, rendering, and linting, creates a feedback loop that leverages model capabilities rather than merely replacing them.
• Interfaces, while potentially ephemeral, are less critical than the underlying "engines" that perform the core tasks. The REPL, in this context, serves as a powerful interface because coding is a strong suit for current AI models.
• Structured reasoning, which involves defining the end state and planning before execution, tends to be more effective than purely tool-based approaches.
• Domain knowledge is paramount and often outweighs the specific choice of tools.
• Evaluation should be matched to the output type: deterministic comparison for objective data and LLM grading for subjective text.
• When an agent appears confused, checking the underlying infrastructure or "plumbing" is often the first step to diagnosing the problem, as apparent reasoning failures can stem from infrastructure bugs.

Campos concluded by emphasizing that 568 commits were made to develop the agent and its supporting infrastructure, highlighting the complexity involved in creating robust AI agents for specialized tasks. The insights gained from this process, particularly the value of REPLs and domain knowledge, offer valuable lessons for the broader AI development community.