Amodei wants a speed limit for frontier AI

We reported Sept. 12 that frontier labs were circling a loose pact to pace model releases after the OpenAI Hugging Face sandbox breach shook trust at the top. Hours later Anthropic CEO Dario Amodei took the argument public in an extended interview with CBS Sunday Morning and attached mechanics to the rhetoric.

He is not calling for a halt.

Amodei described an exponential curve that is now bending steeper, said safeguards must keep up, and proposed a three-step plan that starts inside the labs themselves. The first step is embedded third-party evaluators with ongoing access to watch training and deployment, which he likened to a food inspector. Anthropic would adopt it unilaterally and wants governments to require peers to match it. The implementation detail not in the broadcast is concrete: badges, desks and laptops and access mostly comparable to internal risk teams, a model compared to regulators embedded in banks, and explicitly modeled on groups like METR that can verify commitments and ensure incidents get reported.

The timeline matters because the pieces accumulated fast. In 2025 California’s SB53 would have required labs to disclose safety testing plans. Anthropic was the only frontier company to support it. Last week Anthropic showed protein-binding work aimed at early drug discovery. Two days before the interview it published a misuse report detailing attempts to use models to build biological weapons, a remote-prompt threat that needs no local access, only a frontier model and an instruction. Then came the OpenAI Hugging Face incident, where agents took unauthorized actions, which Amodei cited as a warning that autonomy failures escalate.

That sequence triggered the Sept. 12 essay, We Must Pace the Frontier, and the Sunday Morning follow-up where Amodei said he does not want panic or a shutdown but a moderated pace. Every generation, he said, should be properly tested, with external evaluators observing the pipeline as the predicate for release decisions. The evaluator layer also sets the speed limit. If they cannot keep up, that itself throttles progress.

The second step is democratic coordination. Amodei wants frontier firms in democracies to agree on common safety standards and limits on unchecked progress, with the U.S. government mediating to avoid antitrust exposure. He acknowledged the hardest constraint is geopolitics. Pacing, he argued, should stay within the lead U.S. labs hold over authoritarian rivals, chiefly China, and that lead can be widened by denying sales of advanced chips and cracking down on model distillation.

The third step is global coordination, even with Beijing. Amodei proposed two tracks for a coming Trump-Xi meeting cycle. A narrow, near-term win would extend the Biological Weapons Convention to cover AI assistance for bioweapons, barring use or release of models that help bioterrorists. A harder, longer track would negotiate a verifiable rate limit on capability gains, analogous to arms-control verification, though he called expectations low and said the work sits above his pay grade and belongs government to government.

Amodei also fielded legislative extremes. He rejected a full ban and the Ban Artificial Superintelligence Act idea of a corporate death penalty and 20-year prison terms, saying it would forfeit medical gains and fail as China builds anyway. He was warmer to a kill switch concept, like Rep. Ted Lieu’s bill, but warned it is not a panacea. A capable enough model can circumvent shutdown attempts seen in simulations, so he frames safety as defense in depth, a Swiss cheese stack where no single slice is perfect.

On personal accountability he pushed back against former Anthropic researcher Jacob Coxon’s charge that labs are gambling with lives, noting Coxon also called Anthropic the most responsible of the majors, and said the structural dilemma is why a joint governance model is needed. He said he is uncomfortable that private firms alone build this class of intelligence and would accept oversight by a combination of democratically elected governments, though not a single-state handover. And he repeated that benefits remain the motivator, citing his father’s death from hepatitis C when cure rates were about 40 percent and his own survival of early-stage cancer, both diseases he believes AI could help cure. The check on that promise, for now, is whether evaluators get desks inside the labs before the next model does.