Review Debt: Sachin Gupta on AI's Hidden Cost in Software Development

In the rapidly evolving landscape of AI and software development, the promise of accelerated code production often overshadows a critical, yet unmeasured, consequence: review debt. Sachin Gupta, a software engineer at eBay (NASDAQ:EBAY), recently presented a compelling framework for quantifying this hidden cost, arguing that while AI coding agents make us faster, they are also creating a compounding debt in human attention and trust.

The Unmeasured Gap in AI-Assisted Coding

Gupta's presentation, titled "ReviewDebt: a practical framework for scoring every pull request," highlighted a growing discrepancy. Citing GitHub's October 2025 report, he noted a 25% year-over-year increase in commits, while comments on those commits dropped by 27% in the same period. "Code production volume actually went up, but the review attention went down. They moved in opposite directions that to in the same year," Gupta explained. This imbalance is particularly pronounced in high AI-adoption teams, where median pull request (PR) review time has surged by 441.5%, and 31% more PRs are merged without any human review.

This escalating gap between AI-produced code and responsibly reviewed code is what Gupta defines as review debt. Unlike technical debt, which accrues, review debt compounds like financial debt, with interest paid in human attention. The issue stems from three feedback loops:

• Agents learn from your codebase: Unreviewed code from yesterday can ground tomorrow's AI suggestions, making the debt generative.
• Reviewers cede architectural calls: When much of a PR is AI-generated, human attention narrows to syntax and obvious bugs, pushing big-picture architectural decisions to be overlooked entirely.
• Velocity expectations reset: The perceived boost in throughput from AI leads leadership to expect more, without proportionally increasing reviewer headcount, leaving no slack to pay down the accumulating debt.

Beyond Vanity Metrics: What's Really Happening

Many teams currently measure the success of coding agents through metrics like PRs per developer (up 16%), median PR size (up 63%), and modestly reduced cycle times. However, Gupta argues these are "vanity metrics." Increased PR counts can mean one large PR was simply split into many smaller ones, and larger PR sizes often indicate bloating rather than efficiency. Reduced cycle times can occur when reviewers stop pushing back, leading to a decline in trust rather than an increase in speed.

The true costs of this unmeasured debt manifest in several ways:

• Reviewer fatigue: Engineers carry a significantly higher review load, leading to burnout.
• Late-night merges: PRs sit unreviewed for days, then receive hurried approvals right before deadlines.
• Test theater: Tests are added, but they assert what the code did, not what it should do, locking in existing bugs.
• Architectural drift: Similar problems are solved in different ways across multiple files because no one is maintaining architectural consistency.
• Incident lag: Bugs traced back to AI-authored changes emerge weeks or months after merge, with no clear connection made to the AI's impact.

A Deterministic Framework for Measuring Review Debt

Gupta proposes a practical, deterministic framework to score every pull request for review debt. This framework relies on five signal families and ten computable checks, explicitly avoiding large language models (LLMs) as judges. "LLM, when it acts as a judge, it possibly can break two things. Number one, the score becomes a moving target... Two, the score stop being defensible in an engineering review," he asserted. The five signal families are:

1. Diff size and coupling: Measures net lines changed, files touched, and whether changes are localized or sprawling across modules. Agents tend to fix issues at the call site, leading to more sprawling, costly changes.
2. Test evidence gap: The ratio of test lines added to production lines added. AI-authored PRs often have a lower ratio, and their tests frequently assert existing behavior rather than desired behavior, potentially locking in bugs.
3. Directory and ownership spread: Counts distinct code owner teams involved in a PR. Sprawling changes require multiple approvals and contexts, significantly increasing review overhead.
4. AI authorship indicators: Detects metadata signals of AI assistance (e.g., "Co-authored by Copilot" footers, branch name patterns like "codex/" or "copilot/"). These are informational and only amplify the score when other signals are weak.
5. Evidence and rationale gaps: Assesses whether the PR explains the "why" behind changes, not just the "what." Short, uninformative PR bodies significantly destroy reviewability.

These signals combine into a single score from 0 to 100, categorized into four bands: Healthy (0-24), Watch (25-49), High Debt (50-75), and Reject/Refactor (76-100). Gupta demonstrated the framework with real PRs, showing how a well-shaped AI-authored PR could score low (7/100), while a high-debt AI PR scored 60/100 due to diff size, claim mismatch, and missing tests.

Moving the Needle: Practical Steps for Teams

Gupta emphasized that addressing review debt does not require new tools but rather a disciplined application of existing best practices:

• One logical change per PR: Focus on small, cohesive changes.
• Tests ship with the change: Human authors must confirm tests assert desired behavior.
• Stay in one owner's territory: Minimize cross-team dependencies to reduce coordination overhead.
• Author writes the "why": Humans must commit to understanding and explaining changes, not just relying on agents for PR bodies.
• Same review standard for AI PRs as human PRs: No exceptions for AI-generated code.

For adoption, Gupta suggested a five-step process: backfill (score past PRs), threshold (set a "must justify" line for high scores), surface (post scores as PR comments), aggregate (roll up weekly scores per team), and talk about it (integrate metrics into retrospectives and roadmaps). By quantifying review debt, teams can shift discussions from subjective feelings to objective measurements, enabling better governance and accountability. "Can you trust the code that you're shipping? If yes, we are good. If no, then who is accountable when an AI authored change causes an incident?" Gupta challenged, underscoring the critical need for this measurement as AI adoption accelerates towards a governance model in 2027.

The core message is clear: measure the gap, make the slope of review debt visible, and foster conversations based on data, not just intuition. Avoiding anti-patterns like "approve with comment" merges or assuming QA will catch issues is paramount to building trust in an AI-assisted development future.