Bayes' rule in practice — Probability & Statistical Inference

What you'll learn

Update a belief with data
Explain base-rate neglect with a screening-test example
Update sequentially so each posterior becomes the next prior
Use the odds form with a likelihood ratio

Update the probability of a hypothesis after a test result, one piece of evidence at a time.

Why this matters

A lab test is 90% sensitive and 95% specific, and a patient tests positive. What is the chance they have the condition? Most people, including many professionals, say about 90%. For a rare condition the right answer can be closer to 15%. Bayes' rule is the tool that gets this right. It drives diagnostic testing, spam filters, fraud alerts, quality inspection and the whole Bayesian approach to statistics that the rest of this module builds.

The rule

For a hypothesis H and observed evidence E:

P(H | E) = P(E | H) × P(H) / P(E)
P(E)     = P(E | H) P(H) + P(E | not H) P(not H)

P(H) is the prior: how plausible H was before the evidence.
P(E | H) is the likelihood: how expected the evidence is if H is true.
P(H | E) is the posterior: the updated belief.

The denominator just makes sure the posterior probabilities of H and not H add to 1.

Worked example: a screening test

Prevalence (prior): 1% of people tested have the condition.
Sensitivity P(+ | condition) = 0.90.
Specificity P(- | no condition) = 0.95, so the false-positive rate is 0.05.

Picture 10,000 people:

               condition (100)    no condition (9,900)
test +              90                 495
test -              10               9,405

Of the 585 positives, only 90 have the condition:

P(condition | +) = 90 / 585 ≈ 0.154

With the formula: 0.90 × 0.01 / (0.90 × 0.01 + 0.05 × 0.99) = 0.009 / 0.0585 ≈ 0.154.

The test is good, but the condition is rare, so false positives from the large healthy group outnumber true positives. Ignoring the prior like this is called base-rate neglect.

Updating again

Today's posterior is tomorrow's prior. If the same patient takes a second, independent test and is positive again, start from 0.154:

P(condition | +, +) = 0.90 × 0.154 / (0.90 × 0.154 + 0.05 × 0.846) ≈ 0.766

A negative result updates too, using P(- | condition) = 0.10 and P(- | no condition) = 0.95. After one positive and then one negative, the probability drops to about 0.019.

This is why screening programmes confirm positives with a second test, and why the order of independent evidence does not matter: you reach the same posterior either way.

The odds form

Bayes' rule is easier to do in your head with odds:

posterior odds = prior odds × likelihood ratio
likelihood ratio for a positive (LR+) = sensitivity / (1 - specificity) = 0.90 / 0.05 = 18

Prior odds of 1 : 99, times 18, gives 18 : 99, which is a probability of 18 / 117 ≈ 0.154. Each independent piece of evidence multiplies the odds by its own likelihood ratio.

In code

Python

def update(prior, p_e_given_h, p_e_given_not_h):
    numerator = p_e_given_h * prior
    return numerator / (numerator + p_e_given_not_h * (1 - prior))

p = update(0.01, 0.90, 0.05)     # positive test
print(round(p, 3))               # 0.154
p = update(p, 0.90, 0.05)        # second positive
print(round(p, 3))               # 0.766

Beyond diagnostics

The same calculation appears everywhere:

Spam filters multiply likelihood ratios for words like "invoice" or "winner".
Quality inspection: a vision system flags a part; how likely is it really defective, given a 0.5% defect rate?
Fraud and security alerts: a 99%-accurate rule on 1-in-10,000 events produces mostly false alarms.

Bayesian vs frequentist, briefly

Earlier modules used frequentist methods: parameters are fixed, and probability describes the data you would get over repeated samples. The Bayesian view treats an unknown parameter as uncertain and describes that uncertainty with a probability distribution, updated with Bayes' rule. Both are mainstream; you will use each where it fits. The next two lessons apply the Bayesian view to rates.

Common mistakes

Confusing P(+ | condition) with P(condition | +). Sensitivity is not the chance a positive is real.
Ignoring the base rate. The prior matters most when it is small.
Treating dependent evidence as independent. Repeating the same test on the same sample does not give two independent likelihood ratios.
Using a negative result's likelihoods for a positive one. For a negative, use 1 - sensitivity and specificity.

Recap

Posterior ∝ likelihood × prior; normalise so the probabilities add to 1.
Rare conditions mean many positives are false, even with a good test.
Update sequentially: each posterior becomes the next prior.
Odds form: posterior odds = prior odds × likelihood ratio.

Try it

In the Do tab, write an update function and apply it to a sequence of test results.

Key terms (6)

prior: How plausible a hypothesis is before seeing the evidence, P(H).
posterior: The updated probability of a hypothesis after the evidence, P(H | E).
Sensitivity: The chance a test is positive when the condition is present, P(+ | condition).
Specificity: The chance a test is negative when the condition is absent, P(- | no condition).
base-rate neglect: Ignoring the prior, so a positive result from a good test for a rare condition is wrongly read as very likely true.
likelihood ratio: How much a piece of evidence multiplies the odds; for a positive test, sensitivity / (1 - specificity).