Explaining uncertainty

Why this matters

Every analysis carries uncertainty: the sample might be unrepresentative, the future might differ from the past, the model is a simplification. Managers and clients still need to make decisions. If you hide the uncertainty, they will be overconfident; if you bury them in caveats, they will ignore you. Explaining uncertainty clearly, in terms of the decision, is one of the most valued skills in statistics, actuarial work and operations research.

Say a range, not just a number

A point estimateA single best guess, which sounds precise but hides how wrong it could be. is a single best guess: "We expect 1,200 claims next quarter." It sounds precise but hides how wrong it could be. A range shows the uncertainty: "We expect about 1,200 claims, very likely between 1,050 and 1,350."

Statisticians often report a confidence intervalA range from a procedure that captures the true value in a stated share of repeated samples, such as 95%., for example a 95% interval. Technically, a 95% confidence interval comes from a procedure that captures the true value in about 95% of repeated samples. In plain language for a non-specialist you can say: "Our best estimate is 1,200; the data are consistent with anything from about 1,050 to 1,350."

For forecasts, a prediction intervalA range for where a future individual value is likely to fall, usually wider than a confidence interval for an average. describes where a future value is likely to fall. It is usually wider than a confidence interval for an average, because individual outcomes vary more than averages.

Use numbers, not vague words

Words like "likely", "possible" or "significant" mean different things to different people. In studies of how people interpret these words, "likely" has been read as anything from about 50% to over 90%. Some organisations fix the meanings: the Intergovernmental Panel on Climate Change, for example, uses "likely" to mean a 66 to 100% probability. Unless your audience shares a defined scale, give the number.

The word "significant" is especially risky. In statistics it means a result is unlikely to be explained by chance alone under a particular test; to a manager it means "large and important". A tiny effect can be statistically significant in a big sample. Say how big the effect is.

Natural frequencies and absolute risk

People understand counts better than percentages and conditional probabilities.

• Natural frequenciesProbabilities expressed as counts, such as 3 out of every 10 projects, which people grasp more easily than percentages.: "3 out of every 10 similar projects ran over budget" is easier to grasp than "a 30% probability of overrun".
• Absolute versus relative riskA change expressed as a proportion of the baseline, such as halving defects, which can sound dramatic without the absolute numbers.: "The new process cuts defect risk by 50%" sounds dramatic. If defects fall from 2 in 1,000 to 1 in 1,000, say both: "It halves defects, from 2 per 1,000 parts to 1 per 1,000."

Worked example: a staffing forecast

An analyst forecasts call-centre volume for next month.

Technical version: "The model's point forecast is 42,000 calls, with an 80% prediction interval of 38,000 to 47,000. The upper tail is heavier due to weather-related surges."

Manager version: "Plan for about 42,000 calls. In 8 out of 10 months like this, volume falls between 38,000 and 47,000. The bigger risk is on the high side, mostly from storms. If we staff for 42,000, we have roughly a 1 in 10 chance of more than 47,000 calls and long waits. Staffing for 45,000 would cost about 6% more and cut that risk substantially. The forecast will tighten once we have the first week's data."

The manager version starts from the decision, gives a range, uses natural frequencies, says which direction the risk lies, offers options with costs and says what would reduce the uncertainty.

Show the uncertainty in charts

• Draw intervals as error bars or shaded bands, not just a line.
• For forecasts, a "fan chart" with darker bands for more likely ranges works well.
• Label what the band means ("80% of outcomes fall in the shaded area").
• Avoid chart styles that imply false precision, such as many decimal places.

Common mistakes

• Reporting a single number as if it were certain.
• Using "significant" without saying how big. Give the size of the effect.
• Relative risk alone. Always give the baseline.
• Too many caveats. Keep the main uncertainty in the headline; put the technical details in an appendix.
• Implying certainty you do not have, or false precision such as "1,203.6 claims".

Quick recap

• Give a best estimate and a range, and say what the range means in plain words.
• Replace vague words with numbers; distinguish statistical significance from importance.
• Use natural frequencies and absolute risk; start from the decision and show uncertainty in charts.

Try it

Open the Do tab and rewrite an uncertain result for a non-specialist, then test it on a listener.