Bank AI Demos Die on Permissions, Not Models

Paden Gayle walked through why a working bank demo still dies on the way to production, in an AI Engineer talk that treats deployment as a permission problem first and a model problem second.

The model was fine, Gayle said, and it still did not ship.

He used a document exception assistant to make the point, a system that reviews client documents and drafts exceptions for an analyst, where security then asks which environment will host it, data owners ask who approved the document use, and risk asks who will review a wrong exception before it moves downstream.

Gayle argued those four approvals decide the design, data, model risk, third party risk and audit, and a forward deployed engineer, meaning an engineer embedded with the customer to turn a prototype into a system that can actually run, has to name the owner for each in week one, confirm allowed use and hosting, and confirm what evidence a reviewer will need to keep it running.

That discovery becomes the spec, Gayle said, and when the customer cannot write the spec he leans on three moves: shadow the analyst to map the real sequence, draft a concrete workflow for review, and shrink scope until it fits controls that already exist, for example summarizing a document the analyst already has before reading a second system, before drafting customer facing text, before sending anything automatically.

Autonomy then follows a ladder, Gayle said, from suggest to draft where the system prepares a highlighted exception and draft form for the analyst to check, to acts with approval where each action needs a person to authorize it, to acts where the system operates within preapproved limits, and each step changes the cost of error and which controls must be proven.

The proof is an evidence pack built in sprint one, Gayle said, with intended use and boundaries, data lineage and use approvals, failure modes and fallbacks, results by segment with the weakest group first, and controls with thresholds and owners, plus preserved inputs, prompts, model version, outputs and reviewer actions, and his Friday test asks whether the data owner, environment admins, sponsor, control owners and risk signer are all named, because four or five yeses suggest a credible path while zero to two means rethink the scope before committing.