WiseDocs Refactor: AI Coding Agents and Tech Debt

In a presentation at the AI Engineer World's Fair, Denys Linkov, Head of AI at WiseDocs, detailed the company's decision to undertake a six-month refactoring of their AI pipeline. This initiative was driven by three core issues: the system was too slow to meet customer demand, it was overly complicated to update, and the legacy codebase, spread across more than ten repositories, was unappealing for developers to work with.

The Challenge: Scaling Medical Claims Processing

WiseDocs processes complex medical claims, which often involve PDF documents exceeding 10,000 pages. This scale presents significant challenges for processing and requires a robust and adaptable AI pipeline. The company aimed to improve its underlying technology to better handle this workload and enhance business operations.

Understanding technical debt in AI development

Linkov began by discussing the concept of technical debt, likening it to financial debt that compounds over time. He emphasized that while taking on technical debt to achieve business goals like acquiring new customers is sometimes necessary, it must be managed rigorously. Introducing unnecessary complexity can quickly outweigh the benefits gained. He noted that even with the rapid advancements in AI engineering, as seen in case studies from companies like Anthropic, Spotify, and Stripe, there's a risk of degrading product quality and maintainability. He pointed to uptime data from leading companies showing reliability below 39% or even 49%, indicating that increased shipping velocity hasn't always translated to improved product stability.

The Refactoring Process and AI's Evolving Role

The refactoring effort, which began in April 2025, involved several key tasks, including evaluating orchestrators and refactoring code. Linkov highlighted that the initial two-month evaluation of orchestrators for their AI pipeline, which involved benchmarking five open-source projects, was done before the widespread availability of capabilities like web search in LLMs. He contrasted this with current methods, where an agentic workflow using tools like Deep Research could perform the same task approximately 90% faster. He cautioned against 'AI psychosis,' where the impressive output of research reports doesn't translate into practical product features.

The presentation also touched upon the advancements in AI coding assistance. Linkov shared an experience where a previous model, GPT-3, took three hours and made ten major mistakes when refactoring a piece of code. In contrast, recent benchmarks with models like Sonnet 4.6 and Opus 4.8 showed significant improvements, with Opus 4.8 achieving the task in a single attempt ('one-shot'). These newer models demonstrated better tool usage, including sub-agent calls and verifiable commands, leading to a process that was far less manual, even if slightly more expensive computationally.

The Impact of the Refactor

The consolidation of ten repositories into a single monorepo resulted in a dramatic increase in development velocity. While the legacy output took six years to reach a certain point, the refactored pipeline matched that output in just 67 weeks. The commit rate increased significantly, and more developers became involved in contributing to the codebase. This improved developer experience and productivity were key outcomes of the refactor.

The Question of 'Why Now?'

Linkov addressed the question of whether it was the right decision to refactor at that time or if waiting for further AI advancements would have been more beneficial. He acknowledged that the tooling and model capabilities are continuously improving, making future refactoring potentially easier. However, he argued that the immediate benefits of addressing technical debt and improving the codebase were crucial for the business, especially to avoid the accumulation of 'legacy code' even within AI-native development.

Conclusion: A Worthwhile Endeavor

Ultimately, Linkov concluded that the monorepo refactor was worthwhile. The company achieved its business goals, including reducing processing time, lowering costs, supporting larger files, and enabling faster feature releases. Furthermore, the refactored codebase proved more attractive to developers, fostering a more positive and productive work environment. He advised that AI native delivery should be viewed as a layered approach, emphasizing the importance of strong models, effective tooling, and robust development practices.