Diagnostics

Why this matters

Software will fit a regression to anything and print confident-looking p-values. Whether those numbers mean anything depends on assumptions the software does not check for you. Diagnostics are how you find out whether the model fits, whether a few points are steering the result, and whether the standard errors can be trusted. In a lab or plant, a diagnostic plot is often where you discover a mislabelled sample, a drifting instrument or a second process you did not know about.

What OLS assumes

For the coefficient estimates and their standard errors to be trustworthy, the classic linear model assumes:

1. Linearity: the mean of y really is a linear function of the predictors (as written in the model, including any transformed or squared terms).
2. Independence: errors from different observations are unrelated (no time drift, no repeated measurements of the same unit treated as separate).
3. Constant variance (homoscedasticity): the spread of the errors is the same everywhere.
4. Normality of errors: mainly matters for small samples and for prediction intervals.

None of these is about the distribution of x or of y by itself. They are about the residuals, e = y - fitted.

The four standard plots

import statsmodels.formula.api as smf
import statsmodels.api as sm
import matplotlib.pyplot as plt

model = smf.ols("strength ~ hours", data=df).fit()
infl = model.get_influence()

plt.scatter(model.fittedvalues, model.resid)          # 1. residuals vs fitted
sm.qqplot(model.resid, line="45", fit=True)           # 2. normal Q-Q
plt.scatter(model.fittedvalues, abs(infl.resid_studentized_internal) ** 0.5)  # 3. scale-location
sm.graphics.influence_plot(model)                      # 4. leverage vs residuals

How to read them:

• Residuals vs fitted. You want a shapeless band around zero. A curve means a missing nonlinear term (try adding x**2 or a log). A funnel that widens to the right means variance grows with the mean.
• Normal Q-Q. Points should follow the line. S-shapes indicate heavy or light tails; a few points far off the line are candidate outliers.
• Scale-location. A flat trend means constant variance. An upward trend confirms the funnel.
• Residuals vs leverage. Points in the corners (high leverage and large residual) are the influential ones.

Also plot residuals against time or run order and against any variable you left out. A trend over run order is a classic sign of instrument drift or operator learning, and it violates independence.

Leverage, outliers and influence

These three words mean different things:

• An outlier has a large residual: the model predicts it badly.
• A point with high leverage has unusual predictor values. Leverage is the diagonal of the hat matrix H = A (AᵀA)⁻¹ Aᵀ, written h_i. Each h_i lies between 0 and 1, and they sum to p, the number of coefficients, so the average is p/n. A common flag is h_i > 2p/n.
• An influential point changes the fit noticeably when removed. It usually needs both leverage and a sizeable residual.

Cook's distance combines the two:

D_i = e_i^2 / (p * s^2) * h_i / (1 - h_i)^2        where s^2 = SSE / (n - p)

It equals the (scaled) total change in all fitted values when observation i is deleted. Values above 1, or far above the rest, deserve a look. Some analysts use 4/n as a gentler screen.

Worked example: a mis-recorded specimen

Twenty-five tensile specimens were cured for 2–8 hours, except one cured for 20 hours whose strength was written down as 52 MPa instead of roughly 100. With p = 2 and n = 25, the leverage cut-off is 2p/n = 0.16.

specimen   hours   leverage   Cook's D
  0..23     2–8    0.04–0.08   < 0.1
   24       20.0     0.79      38.4

Specimen 24 sits far to the right in x, so it has huge leverage; because its y is wrong, it drags the line down and gets a Cook's distance of 38. The fitted slope with it is about 0.54 MPa per hour; without it, about 2.96 (close to the true 3). One bad record changed the conclusion.

What should you do?

1. Check the record, not the statistics. Is there a lab notebook entry, a calibration log, a photo of the broken specimen?
2. If it is a recording error, correct it, document the correction and refit.
3. If the value is real, it is telling you something: maybe strength levels off after long cures, so a straight line is the wrong model. Do not delete it just because it is inconvenient.
4. Report results with and without the point when you cannot resolve it.

Fixing common problems

Common mistakes

• Testing normality of y instead of the residuals. A bimodal y can come from a binary predictor and be perfectly fine.
• Deleting points with large residuals until the fit looks good. That is data dredging; it makes p-values meaningless.
• Relying on formal tests alone. With thousands of rows, a normality test rejects trivial departures; with ten rows, it misses serious ones. Look at the plots.
• Forgetting that leverage depends only on x. A high-leverage point that lies on the line is harmless, and it actually improves precision.

Quick recap

• Diagnose with residual plots: vs fitted, Q-Q, scale-location, leverage, and run order.
• Leverage h_i measures unusual x; Cook's distance measures how much one point moves the fit.
• Investigate influential points; correct errors, keep real data and say what you did.

Try it

In the Do tab you will compute leverages from the hat matrix and Cook's distance for the tensile data, and confirm that one specimen dominates the fit.