Communicating model results

Why this matters

A model nobody understands does not get used, and a model people misunderstand gets used for the wrong things. Data scientists and ML engineers spend a lot of their time explaining: to a product manager deciding whether to launch, to a compliance reviewer asking how errors affect customers, to the engineer who will maintain the model next year. The standard written artefact is the model card: a short document that says what the model does, how well, for whom, and where it should not be used.

The core idea: a model card

Model cards were proposed by Mitchell et al. in "Model Cards for Model Reporting" (2019) and are now common practice; Hugging Face shows one on every model page. A good card is one or two pages and covers:

1. Model details: name, version, date, owner and contact, model type (for example "gradient-boosted trees, LightGBM 4"), training code commit.
2. Intended use: the decision it supports, the intended users, and out-of-scope uses ("not for credit decisions", "not validated for patients under 18").
3. Data: where training and evaluation data came from, time period, size, known gaps, and how labels were produced.
4. Metrics: the main metric with its uncertainty, a baseline for comparison, and the decision threshold if there is one.
5. Breakdown by group: performance for relevant subgroups (region, device, age band, language), as practised in the fairness lesson of Machine Learning Foundations.
6. Limitations and risks: known failure modes, what happens when the model is wrong, and the mitigations (human review, fallbacks).
7. Maintenance: how it is monitored, when it will be retrained, and who to tell about problems.

Translate metrics into consequences

Stakeholders do not act on "AUC 0.87". They act on what the model will do to their numbers. Turn metrics into counts and costs at the chosen threshold.

Worked example

A churn model flags customers for a retention call. On 10,000 recent customers, 800 actually churned. At the chosen threshold the model flags 1,000 customers, 480 of whom churned.

precision = 480 / 1,000 = 48%   (about half of the calls reach someone at risk)
recall    = 480 / 800   = 60%   (we reach 6 in 10 of the customers who would leave)
random calling of 1,000 customers would reach about 80 churners (8%)

The plain-English version: "Each month the team can call 1,000 customers. Calling the ones this model flags reaches about 480 customers who would otherwise leave, six times as many as calling at random. It still misses about 4 in 10 churners, mostly customers in their first month, for whom we have little data."

That paragraph states the decision, the gain over a baseline, the main limitation and who is affected. It is worth more than a page of metrics.

Explaining what drives predictions

• Global explanations: which features matter most overall. Use permutation importance or mean absolute SHAP values, and present the top five or so in plain words ("recent support tickets", not n_tix_30d).
• Local explanations: why this one prediction. SHAP values show which features pushed one customer's score up or down. Useful for reviewers and for debugging.
• Be careful with causal language. "Customers with more support tickets are flagged more often" is accurate. "Support tickets cause churn" is a claim the model cannot support.

Communicating uncertainty

• Give ranges, not just points: "AUC 0.87 (95% interval 0.85 to 0.89, from bootstrap resampling)".
• Say how the evaluation was done: time period, holdout or cross-validation, and whether the test data came after the training data.
• Say what would change the conclusion: a new product launch, a pricing change, a different customer mix.

Formats that work

• One-page summary first: decision, recommendation, key number, main risk. Details go in an appendix or the full card.
• Charts with one message each: precision and recall against the threshold, performance by group, calibration (do predicted 30% risks churn about 30% of the time?).
• Versioned in the repository next to the code (MODEL_CARD.md), or in the model registry, so it is updated with each release.

Common mistakes

• Reporting only the best-looking metric, or a metric on training data.
• Leaving out a baseline, so nobody can tell if the model is any good.
• Omitting out-of-scope uses, so the model is reused for a decision it was never validated for.
• Hiding subgroup gaps instead of stating them with a plan.
• Jargon without translation ("F1 of 0.53") for a non-technical audience.
• Writing the card once and never updating it.

Quick recap

• A model card states the model's purpose, data, metrics, subgroup performance, limitations and maintenance plan, including what it must not be used for.
• Translate metrics into counts, costs and comparisons with a baseline.
• Explain drivers with permutation importance or SHAP, without causal claims.
• Report uncertainty, and keep the card versioned with the model.

Try it

Open the Do tab. You will write a one-page model card for a model you built in this course (or a public one), including a plain-English summary for a non-technical reader.