Geoffrey Hinton 10% chance AI kills humans

Ten percent is not unreasonable. That is how Geoffrey Hinton assessed the risk that AI could kill all humans within a decade in a BBC Politics interview.

StartupHub data

Anthropic

Anthropic is an AI safety and research company building reliable, interpretable, and steerable AI systems, best known for the Claude family of models.

Founded

2021

Location

San Francisco, California, USA

Valuation

Private / $100B+ est

Hinton is the British-Canadian computer scientist known for work on artificial neural networks which earned him the title the Godfather of AI.

He shared the 2024 Nobel Prize in Physics with John Hopfield for foundational discoveries that enable machine learning with artificial neural networks.

He called the number very hard to estimate. We have never created beings that may soon be smarter than us, so nobody knows how to give a sensible estimate.

Still, he said it would be very foolish to say 1%. A 10% chance seemed not an unreasonable estimate to him.

That reply was framed by Victoria Derbyshire around a warning from a top Anthropic safety researcher. The researcher had said there is more than a 10% chance AI could cause human extinction within the next decade.

Asked how it could kill us all, Hinton said there are countless ways if it is much smarter than us. It is not worth speculating on each one.

It does not even have to act in the real world to cause devastation. Talking to people alone could create complete chaos.

The technical vectors he named were concrete. It could design very nasty biological viruses, as well as computer viruses, and carry out devastating cyber attacks.

That makes the attacker requirement remote rather than physical. Persuasion and synthesis are enough.

His prescription was to design systems so they will not want to harm us. Instead big tech is racing to make models more intelligent because that is where short term profits are.

His message to world leaders was to regulate, with different solutions for different risks. There is no single fix apart from stopping development, which will not happen because the uses in healthcare, education and drug design are already wonderful.

He finds a chatbot very useful in everyday life. Me with a chatbot is much smarter than just me.

On oversight he conceded a failure point. Britain has a very good AI Safety Institute, but Anthropic did not submit its latest model for testing before release, which he agreed undercuts its purpose.

For international collaboration he used a Cold War analogy. America and the Soviet Union collaborated to prevent nuclear war because interests aligned. On preventing AI takeover, he said all countries share that alignment and will eventually collaborate.

Whether they do it in time is open. He said estimates of how much time we have keep getting shorter, from 30 to 50 years, then 10 to 20, now maybe 10 years or less, with others at Anthropic thinking sooner.

To skeptics who call this a stunt he offered a test. Just use the chatbot. If it can answer any question you give it pretty well, it clearly understands and can reason, and leading mathematicians now say there is no future for those who do not work with them.