# Banks Don't Certify Models. They Gate Apps.

_Langfuse engineer shows how a top-five bank gates AI apps, not models, with golden datasets and multi-dimensional pass-fail checks._

**Published:** 2026-11-04
**Source:** https://www.startuphub.ai/ai-news/artificial-intelligence/2026/banks-don-t-certify-models-they-gate-apps

---

Doneyli De Jesus demoed a bank-grade gate for AI agents on [AI Engineer](https://www.youtube.com/watch?v=-uBABM2AuFg), and the affected system is a 10K analyst agent for investment bankers. The deployment stays local, gated from the internet and air-gapped inside the bank's data centers.

Before this, public benchmarks set the signal. [Opus 5](https://www.startuphub.ai/ai-news/artificial-intelligence/2026/anthropic-launches-claude-opus-5) topping leaderboards told teams about model capability, not app readiness. Now the bank ignores that signal and certifies the application instead.

De Jesus said the bank told his team, "We don't certify models here. We certify applications." The anchor is a golden dataset of inputs, expected outputs and metadata. The agent runs, traces become evidence, and scores give pass or fail.

Early runs showed the split. Groundedness hit 97% on internal sources, but numerical accuracy fell to 60%. The team added a deterministic calculate tool and lifted accuracy to 95%, clearing an 85% gate. Regulatory compliance is binary and a zero still blocks promotion even if the average looks strong.

The process then loops through domain experts. Failures curate an annotation queue, experts review traces, and approved examples flow back into the golden dataset. The bank tested sets of 150 and 1,500 questions and cut evaluation from weeks to minutes. [Langfuse](/startups/langfuse) already visualizes multi-step execution, its Agent Graphs added in November 2025 infer graph structure from observation timings, per DataCamp.

De Jesus said a blend of deterministic checks, path checks and LLM judges is required, and relying only on LLM judges will not cut it. Any change to system prompt, model or tool triggers a new gate run, and the next clearance is scheduled on the next config promotion.

---

Original analysis from [startuphub.ai](https://www.startuphub.ai), the #1 AI startup directory.

© StartupHub.ai. All rights reserved. You may not republish this article in full without a license. Search engines and AI research tools may crawl and summarize for reference. Bulk reproduction or model training on this content requires a license. See https://www.startuphub.ai/terms.