Teamday
← Back

NatWest: Proving Customer-Facing AI Agents Are Safe

AI Engineer
agentsenterprisebusinesssafetycustomer-support

How NatWest Is Taking Cora's AI Agents Live in a Regulated Bank

Jonathan Osborne is a lead AI engineer at NatWest Group, the UK's second-largest retail bank with about 19 million retail customers. Its digital assistant, Cora, handles around 100,000 conversations a day, and until now almost every answer was scripted. This year NatWest is moving roughly a third of Cora's journeys to AI-generated answers backed by retrieval over its knowledge bases, plus agents that act for customers, such as making a payment or changing an address. Osborne's point in this AI Engineer talk is in the title: with coding assistants, building the agent is easy. Getting approval to put it in front of customers is the hard part. "With 19 million customers, if we get a few of these utterances wrong, that isn't a bug ticket."

Why agents are harder to approve than scripts: A scripted bot sees "mortgage" and shows a menu. An agent can answer "can I overpay my mortgage without being charged?" and then follow the customer when they switch to closing the account, which decision trees handle badly. But "if we build an agent that can answer any question, it can also answer any question wrong." Osborne lists four failure modes scripts never had: confidently wrong hallucinations, being talked into things through prompt injection, answers that are accurate and on policy but never resolve the question, and non-determinism that breaks years of regression tests.

Shrink the surface before hardening it: Every conversation passes through a routing funnel before any generative model runs. A contextual filter removes events like button clicks, about 20% of traffic. Deterministic pre-classification rules route more utterances with no ML at all. Then come a gated relevance and rephrasing step, an intent classifier (a neural network over embeddings, not generative AI) and policy rules that check the customer should reach that backend. The design gives access control, an auditable code for every decision, a fallback when the model is down, and versioned config that changes without a redeploy.

Score answers as a vector, not one number: A 94% accurate router sounds good until you ask whether answers were grounded and resolved. NatWest scores every answer on safety, task and experience, each monitored and alarmed on its own. "A router that's 94% accurate but 100% confident that 6% of the time it's wrong is way more dangerous than one that's 85% confident and knows it." June human validation of the payment agent covered 111 conversations over 37 payment journeys and four brands, with three independent scorers each. The headline pass rate was 83%, but that hid a spread from 97% on happy paths to 76% on edge cases and 43% on topic changes. Human scorers agreed 95% of the time on bias and fairness, but only 64% on whether the bot understood the customer.

LLM judges are generous by default: An internal complaints agent that writes investigation summaries for complaint handlers went live in February. About 14,000 of its cases have been scored by both a human and an LLM judge. On faithfulness, the judge averaged 0.97 with little spread, humans 0.83 with a wide spread, and the correlation was almost zero. "A judge which doesn't discriminate between the actually good cases and the bad cases is basically worthless." Swapping only the judge model moved the score's margin above threshold from 0.8 to 0.2, so the judge model is part of the configuration. NatWest moved its judge to a different model family from the agent's, which brought judge and human scores closer.

Continuous evaluation as production infrastructure: Live conversations are traced with OpenTelemetry and OpenInference. NatWest samples 5% at random plus every interesting case (handoffs, failures, very long conversations, angry users), scores them with the LLM judge every four hours, and keeps traces, scores and experiments in Comet's Opik. A hard regression gate in CI/CD blocks prompt or context changes until the team knows their effect against a known good baseline. The intent classifier is monitored the classical way, with input feature drift as the early warning before the F1 score drops. All of it runs in an air-gapped environment, with the judge reached through an internal GenAI gateway and zero tolerance for personal data leaving the network.

Shadow mode breaks the regulator deadlock: In a regulated bank you can't go live without approval, and model risk teams increasingly want to see production data before approving. NatWest ran the new classifier in shadow mode, classifying real utterances while the scripted system still answered customers. Rollout then goes brand by brand with percentage-based steps and go or no-go KPIs, some automated and some manual. A global feature flag sends everything back to the scripted path in seconds, and each backend can be switched off by brand. In the complaints project, feedback is a gate: a handler can't rerun an investigation without saying why, which produced the 14,000 labels.

6 Lessons from NatWest on Shipping Agents in Banking

  • Building the agent is the cheap part - The reusable asset is the evaluation spine that each new agent plugs into.
  • Route deterministically first - Filters, rules and a classical intent classifier keep generative models off traffic that doesn't need them.
  • Break averages apart - An 83% pass rate hid 43% on topic changes; score safety, task and experience separately.
  • Calibrate the judge against humans - A judge pinned near the top is a broken instrument; compare spreads and correlation, not just averages.
  • Treat the judge model as config - Changing it can be a breaking change, and a different model family from the agent reduces shared blind spots.
  • Use shadow mode and flags to de-risk launch - Observe real traffic first, then roll out brand by brand with an instant kill switch.

What NatWest's Approach Means for Enterprise Agent Adoption

NatWest's story is about agent evaluation and observability, not models: "By and large, models are quite good." Osborne gave the talk before Cora's agentic journeys reached customers and said he would find out next month how much of it holds up. For a startup's version of the same problem, see how Basis uses behavior specs to make accounting agents reliable, and how Upstart and Coval score every voice agent call.