AI Engineer captured Tariq Shaukat saying agents lifted lines of code 2,450% and commits 240% while releases rose just 30%, according to research he cited last month. That gap is the news.
Humans cannot close it by watching harder. Shaukat, chief executive of Sonar, argued human in the loop does not scale in the agentic era and pointed to cognitive surrender. Humans agreed with AI when it was right 93% of the time, he said, and four out of five times when it was wrong, per a study from a couple months ago he referenced.
Agents also struggle when the codebase gets large. Shaukat cited a code translation study where accuracy was 91% at 10,000 lines and collapsed by 50,000 lines, and a second study where tokens needed for the same task varied 30x as the agent burned budget trying to map structure. Errors compound too. He pointed to Meter data where Claude Mythos preview completed 16-hour human tasks at 50% accuracy, with the curve flattening sharply at 80%.
And the judge is shaky. Shaukat said PayPal AI Research found 7% to 17% of verdicts changed across identical replications depending on model and situation this month, per his account, and a Meta study found a simple "are you sure?" flipped answers 7% to 34% of the time. His response is a code confidence formula built on intent, guide rails not guard rails, independent verification, and auditable evidence. He contrasted guide rails as nudges that keep agents on track with guard rails that only catch them after they leave the road.
For context, Sonar sells verification as the factory floor. Its SonarQube product is pitched as the gold standard check, while Sonar Vortex helps your coding agent generate more maintainable, reliable, and secure code faster and more efficiently, per Sonar documentation. That documentation also notes Sonar Vortex and the Remediation Agent each require a separate subscription in addition to an annual Team plan or Enterprise plan in SonarQube Cloud. Shaukat said customers using Vortex in long-running tasks on large codebases saw a 36% reduction in token cost and 92% reduction in issues in the commit, again by putting the agent closer to the answer. In a survey of more than 1,000 customers he cited, SonarQube users were 81% more likely than non-users to report a very positive impact on code security from AI coding, 34% more on time to market and 76% on outage frequency.
The numbers are directional, not controlled trials, and Shaukat did not disclose sample composition, task mix, or baseline quality gates. His preview of Sonar’s forthcoming State of Code survey, 5% running agentic software factories today and 34% expecting to within a year, 39% in financial services, shows intent rather than verified deployment. A clearer read would need independent replication on shared repositories with cost per merged pull request and post-release defect rates. Until then the open question is whether factories engineered for confidence can turn abundant code into shippable releases without reintroducing the rework Toyota set out to eliminate.
