Tier models by risk and document their limits, approvals and oversight with model cards and risk records.
Why this matters
Models make or shape decisions about people: who gets a loan, which job applicant is shortlisted, which patient is flagged for follow-up. When those decisions go wrong, "the model did it" is not an acceptable answer to a customer, a regulator or a court. Model governance is the set of records, reviews and controls that let an organisation say what a model does, who approved it, what its known limits are and how it is watched. For a machine learning engineer, it is part of the job, not paperwork someone else handles.
The core idea: risk-based controls
Not every model needs the same scrutiny. A model that sorts internal support tickets carries less risk than one that sets credit limits. Most frameworks scale controls by impact (how much harm a wrong decision can do, and to whom) and autonomy (does a human review the output, or does it act directly?).
A practical tiering:
| tier | example | typical controls |
|---|---|---|
| low | internal ticket routing, demand forecast for one store | model card, standard tests, owner on record |
| medium | marketing targeting, product recommendations | plus fairness checks, monitoring with alerts, sign-off from the product owner |
| high | credit, hiring, insurance pricing, medical triage | plus independent validation, legal and compliance review, human oversight, documented appeal route, periodic re-approval |
Documenting a model
Two widely used formats:
- Model card: what the model is for (and not for), training data summary, evaluation results overall and per relevant subgroup, known limitations, ethical considerations, owner and version. You met these in Applied Machine Learning.
- Datasheet for datasets: where the data came from, how it was collected, consent and licensing, known gaps and biases, retention.
Add a model risk record for anything medium or high tier: the risk tier and why, the decision it supports, failure modes and their impact, mitigations, the monitoring plan, and the approvals with names and dates.
Approvals and the model lifecycle
A governed lifecycle has explicit gates:
- Proposal: use case, tier, data sources, legal basis for using personal data.
- Development: experiments tracked; data lineage recorded.
- Validation: for high-tier models, someone who did not build it reproduces the results, challenges assumptions and tests robustness and fairness.
- Approval: named owners sign off (business owner, ML lead, and for high tier risk/compliance).
- Deployment: the registry records which version is live and when it changed; rollback is tested.
- Monitoring and review: scheduled re-reviews (for example yearly), plus a re-review after significant drift or a change of use.
- Retirement: decommission cleanly, keep records for the required period.
A model registry (such as the MLflow Model Registry) supports this by storing versions, stages and annotations, but the approvals themselves are a human process.
Fairness, explainability and oversight
- Fairness: compare error rates and outcomes across relevant groups (for example false positive rates by age band). There are several fairness definitions and they can conflict; document which one you chose and why.
- Explainability: be able to explain the main drivers of a decision at the level the audience needs. Global feature importance for reviewers; reason codes for an individual decision where required (credit decisions in many countries require reasons).
- Human oversight: decide where a person reviews outputs, can override them, and how a person affected by a decision can contest it.
Frameworks and law
You do not need to memorise them, but you should know they exist and where to read them:
- NIST AI Risk Management Framework (AI RMF 1.0): a voluntary US framework organised around Govern, Map, Measure and Manage. See www.nist.gov/itl/ai-risk-management-framework (checked on 2026-10-23).
- SR 11-7: US banking supervisory guidance on model risk management, long the reference for validation in finance.
- EU AI Act: risk-based EU regulation that places extra obligations on "high-risk" systems such as those used in hiring or credit.
- Privacy law such as GDPR, which covers personal data used in training and logging.
Requirements differ by country and sector and change over time. Work with your legal and compliance teams; this lesson is an orientation, not legal advice.
Common mistakes
- Documentation after launch. Writing the model card from memory months later.
- One-size-fits-all process. Heavy review for every small model, so teams route around it.
- Approval by the builder. High-impact models need independent validation.
- Ignoring changes of use. A model approved for marketing quietly reused for credit decisions.
Quick recap
- Governance scales controls with impact and autonomy: tier your models.
- Keep a model card, a datasheet and, for higher tiers, a risk record with named approvals.
- Gate the lifecycle: proposal, validation, approval, monitoring, periodic review, retirement.
- Know the main frameworks (NIST AI RMF, SR 11-7, EU AI Act) and involve legal and compliance.
Try it
Open the Do tab and take the quiz to practise tiering models and choosing the right controls.