Describe data, reason with probability, and draw honest conclusions from samples, with short fill-in-the-blank Python exercises.

Statistics Fundamentals

## Why this matters

Probability is the language statistics uses to talk about uncertainty. A quality engineer asks how likely a batch is to contain a defect, an actuary prices the chance of a claim, a data scientist estimates how often a model is wrong. You do not need advanced math for most of it. Four rules, applied carefully, cover a surprising amount of real work.

## The basics

An **experiment** is anything with an uncertain result: rolling a die, testing a part, a customer visiting a page. The **sample space** is the set of all possible outcomes. An **event** is a set of outcomes you care about.

When all outcomes are equally likely:

```text
P(event) = (number of outcomes in the event) / (total number of outcomes)
```

Rolling a fair six-sided die, P(even) = 3/6 = 0.5, because {2, 4, 6} are three of six outcomes.

Probabilities are always between 0 (impossible) and 1 (certain). You can write them as fractions, decimals or percents: 1/4 = 0.25 = 25%.

When outcomes are not equally likely, use **relative frequency**: the share of times the event happened in a long record. If 37 of the last 1,000 shipped units were returned, P(return) is about 0.037.

## Rule 1: the complement

The probability that an event does **not** happen is one minus the probability that it does.

```text
P(not A) = 1 - P(A)
```

This is often the easy way in. "What is the chance of at least one defect in 3 parts?" is hard directly but easy as `1 − P(no defects)`.

## Rule 2: addition (A or B)

```text
P(A or B) = P(A) + P(B) - P(A and B)
```

You subtract `P(A and B)` because outcomes in both events were counted twice.

Worked example: draw one card from a standard 52-card deck. What is P(heart or king)?

```text
P(heart) = 13/52
P(king)  = 4/52
P(heart and king) = 1/52     (the king of hearts)
P(heart or king) = 13/52 + 4/52 - 1/52 = 16/52 ≈ 0.308
```

If two events cannot both happen (they are **mutually exclusive**), `P(A and B) = 0` and the rule becomes plain addition. Rolling a 1 or a 6 on one die: 1/6 + 1/6 = 1/3.

## Rule 3: multiplication for independent events (A and B)

Two events are **independent** when one happening does not change the probability of the other. Separate coin flips, separate dice, parts from a stable process.

```text
P(A and B) = P(A) * P(B)        (independent events only)
```

Worked example: a process makes 2% defective parts, independently. Pick 3 parts.

```text
P(no defect in one part) = 0.98
P(no defects in 3 parts) = 0.98 * 0.98 * 0.98 ≈ 0.941
P(at least one defect)   = 1 - 0.941 = 0.059
```

That combines the multiplication rule with the complement rule, the most common pattern in practice.

## Rule 4: dependent events

If events are **not** independent, multiply by the probability of the second event **given** the first:

```text
P(A and B) = P(A) * P(B given A)
```

Draw two cards without putting the first back. P(both aces) = 4/52 × 3/51 ≈ 0.0045, because after one ace is gone only 3 aces remain among 51 cards. The next lesson is all about this "given" idea.

## Checking with simulation

When a calculation feels slippery, simulate it. Python's `random` module with a fixed **seed** gives the same "random" results every run, which makes results reproducible:

```python
import random

random.seed(42)
trials = 100_000
hits = 0
for _ in range(trials):
    a, b = random.randint(1, 6), random.randint(1, 6)
    if a == 6 or b == 6:
        hits += 1
print(hits / trials)   # close to 11/36 ≈ 0.306
```

The exact answer: P(at least one six in two rolls) = 1 − (5/6)² = 11/36 ≈ 0.306. The simulation agrees to two or three decimal places, and more trials bring it closer.

## Common mistakes

- Adding probabilities of events that overlap without subtracting the overlap.
- Multiplying probabilities of events that are not independent.
- Treating "three heads in a row" as making tails "due". Independent flips have no memory: this is the gambler's fallacy.
- Getting a probability above 1 or below 0. That is always a sign of an error.

## Quick recap

- Equally likely outcomes: favourable / total.
- Complement: P(not A) = 1 − P(A).
- Addition: P(A or B) = P(A) + P(B) − P(A and B).
- Multiplication: P(A and B) = P(A) × P(B) when independent; otherwise use P(B given A).
- Simulate with a fixed seed to check your reasoning.

## Try it

In the Do tab, write the addition rule and compare it with a simulation.


Compute simple and combined probabilities with the complement, addition and multiplication rules.

Probability rules

Probability Part 1: Rules and Patterns: Crash Course Statistics #13

Addition rule for probability | Probability and Statistics | Khan Academy

## Why this matters

A screening test is "99% accurate" and your result is positive. What is the chance you actually have the condition? Most people, including many professionals, guess about 99%. The real answer is often far lower. Medical lab staff, data scientists building classifiers, and quality engineers reading inspection results all need to get this right. The tool is **conditional probability**.

## What "given" means

`P(A | B)` is read "the probability of A **given** B": the probability that A happens if we already know B happened. Knowing B shrinks the world we are looking at to only the cases where B is true.

```text
P(A | B) = P(A and B) / P(B)
```

Worked example. A class of 100 students: 40 study statistics, 25 study chemistry, and 10 study both.

```text
P(chemistry | statistics) = P(both) / P(statistics) = (10/100) / (40/100) = 10/40 = 0.25
P(statistics | chemistry) = (10/100) / (25/100) = 10/25 = 0.40
```

Notice that `P(A | B)` and `P(B | A)` are **different numbers**. Mixing them up is the single most common error in this topic, and it has a name: confusing the inverse.

## Independence, restated

Events A and B are independent exactly when knowing B does not change A: `P(A | B) = P(A)`. In the class, P(chemistry) = 0.25 and P(chemistry | statistics) = 0.25, so in this made-up class the two choices happen to be independent.

## Test results: the four boxes

A diagnostic or inspection test has two properties you can measure in a lab:

- **Sensitivity** = P(test positive | condition present). How well it catches real cases.
- **Specificity** = P(test negative | condition absent). How well it clears healthy cases.

And one property of the population:

- **Prevalence** = P(condition present) in the people being tested.

What the patient wants is the **positive predictive value (PPV)** = P(condition present | test positive). That is the inverse of sensitivity, and it depends heavily on prevalence.

## Worked example with counts

The easiest way to compute PPV is to imagine 10,000 people. Prevalence 1%, sensitivity 99%, specificity 95%.

```text
Have the condition:     1% of 10,000 = 100
  test positive (99%):  99      true positives
  test negative:         1      false negative
Do not have it:                 9,900
  test negative (95%):  9,405   true negatives
  test positive (5%):     495   false positives

All positives = 99 + 495 = 594
PPV = 99 / 594 ≈ 0.167
```

Only about 1 in 6 positives is real. The test is good, but the condition is rare, so false positives from the large healthy group swamp the true positives from the small sick group. This is why positive screening results are usually followed by a second, more specific confirmatory test.

## Bayes' rule

The same calculation as a formula:

```text
P(D | +) = P(+ | D) * P(D) / P(+)

where P(+) = P(+ | D) * P(D) + P(+ | not D) * P(not D)
           = sensitivity * prevalence + (1 - specificity) * (1 - prevalence)
```

Plugging in: `0.99 × 0.01 / (0.99 × 0.01 + 0.05 × 0.99) = 0.0099 / 0.0594 ≈ 0.167`. Same answer.

Change only the prevalence to 20% (for example, testing people who already have symptoms) and the PPV jumps to about 0.83. The test did not change; the population did.

## Beyond medicine

- A spam filter flags an email. P(spam | flagged) depends on how much of your mail is spam to begin with.
- A fraud model flags a transaction. If 1 in 1,000 transactions is fraud, even a strong model produces mostly false alarms.
- An automated inspection rejects a part. If the process is very good, many rejects may be good parts.

## Common mistakes

- Treating sensitivity as if it were PPV (confusing the inverse).
- Ignoring the base rate (prevalence). This is called base-rate neglect.
- Forgetting that PPV changes when you test a different population.

## Quick recap

- `P(A | B) = P(A and B) / P(B)`; it is not the same as `P(B | A)`.
- Sensitivity and specificity describe the test; prevalence describes the population.
- PPV = true positives / all positives. Rare conditions mean many false positives.
- Bayes' rule turns P(+ | D) into P(D | +).

## Try it

In the Do tab, write the PPV formula and see how it changes with prevalence.


Use conditional probability and Bayes' rule to work out what a positive test result really means.

Conditional probability

Bayes theorem, the geometry of changing beliefs

Intro to Conditional Probability

## Why this matters

Measurement errors, part dimensions, test scores and many biological measurements pile up in a familiar bell shape. That shape, the **normal distribution**, lets you answer practical questions from just two numbers: "What fraction of bolts will be out of tolerance?", "Is this lab result unusually high?", "What score puts someone in the top 5%?" It also sits underneath the confidence intervals and hypothesis tests later in this course.

## The shape

A normal distribution is symmetric and bell-shaped, centred on its mean `μ` (mu), with spread set by its standard deviation `σ` (sigma). Mean, median and mode are all equal. The curve never quite touches zero, but values far from the mean are very rare.

Two normal curves with the same mean and different SDs have the same centre; the one with the bigger SD is wider and flatter. The total area under any probability curve is 1, and the **area between two points is the probability** of landing between them.

## The 68-95-99.7 rule

For any normal distribution:

- about **68%** of values lie within 1 SD of the mean,
- about **95%** within 2 SD,
- about **99.7%** within 3 SD.

Example: adult resting heart rates in a study are roughly normal with mean 72 bpm and SD 8 bpm. About 95% of people are between 72 − 16 = 56 and 72 + 16 = 88 bpm. A reading of 100 is 3.5 SD above the mean, which is very unusual.

## z-scores

A **z-score** says how many standard deviations a value is from the mean:

```text
z = (x - mean) / sd
```

- z = 0: exactly at the mean.
- z = 1.5: one and a half SDs above.
- z = −2: two SDs below.

z-scores let you compare values on different scales. A score of 82 on a test with mean 70 and SD 6 has z = 2.0. A score of 88 on a test with mean 80 and SD 8 has z = 1.0. The 82 is the more impressive result.

Turning values into z-scores is called **standardising**. Every normal distribution becomes the **standard normal**, with mean 0 and SD 1, so one table (or one function) answers every question.

## From z to probability

The **cumulative distribution function (CDF)** gives P(Z ≤ z), the area to the left of z. Some values worth knowing:

```text
z     P(Z <= z)
-2    0.023
-1    0.159
 0    0.500
 1    0.841
 1.645 0.950
 1.96 0.975
 2    0.977
```

Three question types:

```text
P(X below a)         = cdf(z_a)
P(X above a)         = 1 - cdf(z_a)
P(X between a and b) = cdf(z_b) - cdf(z_a)
```

## Worked example: tolerance

A machine cuts rods with mean length 50.0 mm and SD 0.2 mm. The spec is 49.6 to 50.3 mm. What fraction is in spec?

```text
z_low  = (49.6 - 50.0) / 0.2 = -2.0    cdf = 0.0228
z_high = (50.3 - 50.0) / 0.2 =  1.5    cdf = 0.9332
in spec = 0.9332 - 0.0228 = 0.9104   (about 91%)
```

About 9% of rods are scrap, mostly on the long side. Centring the process at 49.95 mm would help more than buying a tighter machine.

## In Python

The standard library has a normal distribution built in:

```python
from statistics import NormalDist

rods = NormalDist(mu=50.0, sigma=0.2)
print(rods.cdf(50.3) - rods.cdf(49.6))   # 0.9104...

z = NormalDist()                          # standard normal
print(z.cdf(1.96))                        # 0.975...
print(z.inv_cdf(0.95))                    # 1.6448...: the 95th percentile
```

`inv_cdf` goes the other way: from a probability to the value that has that much area to its left.

## When the normal model does not fit

Not everything is normal. Incomes, wait times and file sizes are right-skewed; counts of rare events are not bell-shaped at all. Always look at a histogram before you use normal probabilities. If the data is clearly skewed, the 68-95-99.7 rule and z-table answers will be wrong, sometimes badly in the tails.

## Common mistakes

- Using normal probabilities on skewed data.
- Forgetting `1 −` when the question asks for "above".
- Dividing by the variance instead of the SD in the z formula.
- Reading the table as the area to the right when it gives the area to the left.

## Quick recap

- Normal = symmetric bell set by mean and SD.
- 68-95-99.7 within 1, 2, 3 SDs.
- z = (x − mean) / sd; the CDF turns z into a probability.
- Below: cdf. Above: 1 − cdf. Between: difference of two cdfs.

## Try it

In the Do tab, write the z-score formula and use it to compute a tolerance yield.


Use z-scores and the normal curve to find the probability of values above, below or between limits.

The normal distribution

The Normal Distribution: Crash Course Statistics #19

Z-Scores and Percentiles: Crash Course Statistics #18

Probability rules, conditional probability and Bayes' rule for test results, and the normal distribution with z-scores.