Matthew Jewkes told AI Engineer that the fix for flaky enterprise agents is full automation with humans only on exceptions.
I think he diagnosed the disease correctly and oversold the cure. Jewkes said agents in the enterprise are a pain to maintain, fail in unusually shaped ways, and drag experts back in through phone notifications that waste scarce judgment. His answer is to engineer agency out of the happy path, use AI to pre-filter inputs and reject adversarial or business-critical cases, and let the automation run untouched while humans handle only the exhaust. That is a clean story until you hear what it took to earn it.
Three and a half nines is not a strategy.
Jewkes built his case on drug indexing work from his biopharma data company, sold to Clarivate in 2023 according to his talk, where Clarivate holds tens of millions of documents with hundreds of thousands of edits per year. His team processed tens of thousands of indexing tasks per month, started with a research-style agent at 80 to 83% accuracy after a few sprints, then compiled 1,000 evals with domain experts to separate critical errors from undecided cases and 3 to 5% true exceptions. Only after routing out ambiguity and escalation cases did standard methods reach 95% accuracy, Jewkes said, and the grind from 95% to 99.95% was pure hill climbing with evals, tricks, and hours reading sycophantic model output. To his credit, he said this outright. Full automation was expensive capex, required trust and ownership changes, and only made sense for scaled processes with written specs and seven figures of revenue behind them.
That caveat is doing most of the work. The classic UX meaning of the happy path is the ideal uninterrupted flow with no errors or edge cases, which product teams contrast with blocking failures that must be fixed first. Jewkes wants AI to enforce that separation by detecting confusion, aborting unusual cases, and sending plausibly business-critical work to a heavily tested pipeline. For that to hold, you need inputs clean enough to route, experts willing to sit with 1,000 evals instead of a blank spreadsheet, and an organization ready to freeze the 20% of work it never really specified. Most growth-stage teams do not have any of the three.
Startups in discovery should keep humans in the loop, Jewkes said, and on that point he is hard to argue with. Do not buy full automation until you can afford the thousand evals and the fight to enforce them.
