Today in AI: Mathematics of AI Uncertainty

Google DeepMind laid out the mathematics of AI uncertainty this week, with Cambridge professor and frontier AI co-lead Zubin Ghahramani arguing that scaling up alone will not deliver reliable intelligence.

His argument is direct. Intelligence is decision making with limited perception, so any useful system must represent, update and act on uncertainty.

Why chatbots sound sure when they are wrong

Ghahramani breaks uncertainty into two types. Aleatoric uncertainty is irreducible randomness, like which way a pedestrian will turn, while epistemic uncertainty is not having seen the situation before, like a horse jumping into a hailstorm in front of a self-driving car.

Current large language models mix the two together. They are trained to predict the next token, not to maintain a coherent belief state.

That explains the familiar failure mode. Ask a model how confident it is and it will answer, push back with "I think you're wrong" and it flips.

The research lineage here is old. Ghahramani points to adversarial examples from more than a decade ago, where imperceptible pixel changes made a school bus read as a cheetah at 99% confidence.

For builders, this matters because correctness without calibration is unusable. In medical or autonomous settings, a confidently wrong answer is worse than no answer.

Semantic entropy, developed by one of Ghahramani's students, tries to infer uncertainty from the model's internal next-token distribution. Low entropy looks spiky and certain, high entropy is spread out. It helps, but Ghahramani calls it faking a calculator by showing it many sums instead of teaching it arithmetic.

Where DeepMind already bets on Bayesian thinking

Weather is the counterexample. GenCast, DeepMind's recent diffusion-based forecaster, does not predict once.

It generates an ensemble of trajectories for events like Hurricane Melissa, then updates that ensemble as new sensor data arrives. That is explicit Bayesian updating, and it runs in about eight minutes versus hours on a traditional supercomputer.

AlphaFold does the same thing visually. Its predicted structures are color-coded by confidence, because physics means some regions genuinely wiggle and the model should say so.

The lesson for product teams is counterintuitive. Adding uncertainty makes predictions more accurate and more actionable.

An ensemble tells an emergency manager not just where a storm might go, but how much to hedge. That is a feature you can price, not a research footnote. Startups selling forecasting, drug discovery or robotics should treat calibrated uncertainty as a differentiator, not a disclaimer.

The real debate over scale versus architecture

Ghahramani frames AGI as two camps. One bets that more data and compute will smooth over gaps. The other, his, bets on architecture.

He worked on neural nets in 1986, reading the parallel distributed processing books for Aravind Joshi at Penn, building small language models on a 65,000-processor Connection Machine in 1989, then leaving nets for Bayesian machine learning with Geoffrey Hinton because the math felt unfinished.

Three gaps now test the scale thesis. Continual learning still fails with catastrophic forgetting, while Bayesian updating in theory does not. Energy is off by orders of magnitude, 20 watts for a brain versus a data center for a large model. Data efficiency is poor compared to a child learning from few examples.

The catch is compute. Ghahramani admits exact Bayesian inference is intractable, often NP-hard, which is why the field abandoned it 15 years ago.

His bet is that today's hardware and decades of approximation methods make a revisit viable. Until then, any claim that scale cures hallucinations should be read skeptically.

Trustworthy AI will not be the one that never says I do not know. It will be the one that knows when to say it.