Jonathan Osborne demoed NatWest's safety stack for Kora on AI Engineer, affecting 19 million retail customers and 100,000 daily conversations. The attack surface is remote. Any customer utterance can carry prompt injection, hallucination, or nondeterministic output.
Before this year Kora was scripted. Osborne said the vast bulk of 100,000 daily answers came from menus. A mortgage overpay question like "Can I overpay without being charged?" hit a keyword, showed options, then queued a human. Now NatWest is moving a third of Kora journeys to retrieval-augmented generation and action-taking agents.
NatWest first shrank what reaches a model. Osborne described a routing funnel that filters about 20% button clicks, then applies deterministic pre-classification rules with no ML. Only survivors hit a gated relevancy and rephrasing step and a neural intent classifier over embeddings, plus policy rules for access control. Every decision carries a code for audit.
Accuracy alone was not enough. Osborne said, "The actual creation of the agent that's kind of the easy part. The much harder part I think is proving that it's safe." NatWest now scores answers on a vector of safety, task completion, and experience, so a 94% accurate router that is 100% confident when wrong is flagged as more dangerous than an 85% router that knows its limits.
Human validation in June tested 111 conversations across 37 payment journeys and four brands, with three scorers each. Osborne reported a headline 83% pass rate that hid a spread from 97% on happy paths to 76% on edge cases and 43% on topic changes, plus 64% scorer agreement on intent understanding versus 95% on bias and fairness. Automating that check with an LLM judge on 14,000 complaint summaries from a February live internal agent showed a mean judge score of 0.97 against a human mean of 0.83 with near zero correlation, and swapping the judge model alone moved a threshold margin from 0.8 to 0.2.
The daily brief
Get the free daily brief with the day's AI news, new AI startups and funding rounds every morning.
Join 64,000 other subscribers.
NatWest captures traces via OpenTelemetry and OpenInference, samples 5% plus handoffs and long or angry chats, scores offline every four hours, and blocks merges on regression against a known good baseline. All traces and scores land in Comet for both generative traces and classical ML drift monitoring, with judges run in an airgapped network over a private path and zero PII leaving the bank. NatWest is UK-focused and serves over 20 million customers across retail, commercial and private banking, per its corporate site, which frames the 19 million retail figure Osborne cited.
NatWest ran shadow deployment before any customer saw AI answers, routing live utterances to the new classifier while the scripted system still replied. Rollout is brand by brand with percentage ramps and feature flags that can route all traffic back to scripts in seconds. The next test is next month, when NatWest goes live and learns whether those metrics hold.
