AI Video

Your Support Team Should Ship Code, Lisa Orr, Zapier

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

3 min read

Your Support Team Should Ship Code, Lisa Orr, Zapier

Lisa Orr, Product Leader at Zapier, shared a compelling narrative about how her company is leveraging artificial intelligence to transform its support operations, enabling the support team to actively ship code.

The core problem was the sheer volume of support tickets generated by API changes, overwhelming traditional support workflows. Zapier's journey began with a clear objective: to expedite the resolution of integration issues by empowering the support team with AI-driven tools.

Your Support Team Should Ship Code, Lisa Orr, Zapier - Video
YouTube Video

The initial approach involved shadowing engineers to understand their workflows and identify opportunities for AI intervention. This led to the creation of an "API playground" equipped with AI tools for diagnosis and test generation. However, this proved ineffective because it disrupted engineers' existing workflows, pulling them away from their integrated development environments (IDEs). Orr highlighted this early misstep, noting that the tools "pulled builders out of their workflows." This realization was a critical pivot point, shifting the focus towards embedding AI capabilities directly into the tools engineers already used.

The next iteration involved developing "MCP tools" designed for direct use within IDEs. While these tools saw some adoption, the most promising, "Diagnosis," was too slow. Engineers were unwilling to wait for its lengthy processing time, underscoring the need for an asynchronous solution. This frustration with synchronous processes is a common hurdle in AI adoption, where immediate feedback is often desired but not always feasible for complex tasks. The team recognized that "Engineers wouldn't wait for it, revealing we needed an asynchronous approach."

This led to the development of the Scout Agent, a sophisticated system designed to chain AI tools together autonomously. The agent's primary function is to read support tickets, gather relevant context, generate potential fixes along with corresponding tests, and then submit these as merge requests (MRs) ready for review. This autonomous capability has significantly boosted the support team's capacity, allowing them to manage high ticket volumes effectively. The ability to present "an MR ready for review means they can validate and ship a fix quickly before needing to jump on the next incoming ticket" is a game-changer for support efficiency.

A fundamental insight gained throughout this process is that the true challenge lies not solely in code generation, but in the surrounding ecosystem. Before Scout Agent can generate code, it must possess the correct context and demonstrate its reasoning to build engineer trust. Post-generation, engineers need a streamlined way to validate and correct proposed fixes; otherwise, MRs risk becoming abandoned. Embedding Scout Agent directly within GitLab addressed this by eliminating context switching, allowing teams to iterate on solutions seamlessly.

Zapier measures improvement through three key failure modes: categorization accuracy (whether Scout should even attempt a ticket), fixability assessment (if a code fix is indeed required), and solution quality (the efficacy of the generated code). Each metric provides distinct avenues for improvement.

Currently, Scout Agent is responsible for 40% of the support team's integration fixes. The future holds expansion into engineering teams and further downstream automation, including testing, shipping, and migration processes.

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsAIAI AgentsCode GenerationLisa OrrOperational EfficiencySaaSWorkflow AutomationZapier

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.