AI Governance and Risk
AI model risk management and assurance
Evaluation harnesses, quality bars, drift monitoring, and audit-ready documentation for regulated workflows
- Duration
- 4 to 8 weeks
- Ladder stage
- Product
- NATIVE stages
- Validate, Expand
The situation
When this engagement is the right one
In a regulated workflow, the question is not whether the model is good. It is whether you can show, months later, that it was good enough, on which cases, judged by whom, and what happened when it was not.
You are probably seeing
- Model quality is asserted from a demo rather than measured against a set.
- There is no held-out set representing the hard cases.
- Nobody monitors override rate, so degradation is invisible until a complaint arrives.
What we do
The work, in the order it happens
- 01Build the evaluation set from real cases, including the hard tail.
- 02Define quality bars per decision type, with the reviewer who set them named.
- 03Stand up an eval harness in CI so regressions block release.
- 04Instrument drift, override rate, and exception volume.
- 05Produce the audit documentation template and populate the first cycle.
What you get, and keep
- An evaluation set and harness, in your repository.
- Documented quality bars per decision type.
- A monitoring dashboard specification with alert thresholds.
- A completed first-cycle audit pack.
Prerequisites
- Access to historical cases, including the ones that went wrong.
Not included
- We do not act as an independent assurance function for regulatory submission.
Duration and price shape
4 to 8 weeks
Fixed fee, scoped by number of decision types.
Where this sits in the method
Commercially this is a Product engagement on the Proof, Product, Platform ladder.
FAQ
Questions we get asked
- Do you build the eval set or do we?
- We build the first one with your reviewers, then hand over the process so your team extends it.
- What is a reasonable quality bar?
- One your reviewers will defend in front of a regulator. We facilitate that conversation; we do not set it for you.
Related engagements
Work that usually sits either side of this
Next step
Ready to scope model risk and assurance?
Bring the pilots you already have running. The first call is a scoping conversation, not a pitch.