Fit and check multiple regression and GLMs, design randomised, blocked and factorial experiments, and turn analyses into clear, reproducible reports for the researchers you work with.

Regression & Experimental Design

Before the meeting, write down three questions you will ask about the question itself, three about how the data were produced, and two about constraints and deliverables.

In the meeting, spend the first 5 minutes only listening and asking clarifying questions. Do not suggest any methods yet.

Restate the question in measurable terms (unit, comparison, smallest important difference) and ask your collaborator to confirm or correct it.

Identify at least one design issue (in the role-play: pseudo-replication from 3 readings per batch, and method confounded with week) and explain it in plain language.

Agree on what you will deliver, in what form and by when, and what the analysis can and cannot conclude.

Within a day, write and send a one-page scope summary: question, data, planned analysis, deliverables, dates, open questions.

Ask your collaborator for one piece of feedback on what was unclear or unhelpful.

## Why this matters

Most statistical work at the mid-level is done **with** someone else: a chemist planning a stability study, a biologist with a plate of assays, a plant manager who wants to know why yields dropped, a product team running an A/B test. The technical skills in this course only help if you understand the real question, the real data and the real constraints. Statisticians and data scientists who are good at consulting get brought in early, when they can still shape the design; those who are not get handed a spreadsheet the week before a deadline.

## What a consulting intake is for

The first meeting (the **intake**) has four goals:

1. Understand the **scientific or business question** in the collaborator's own words.
2. Understand **how the data were or will be produced**: units, treatments, measurements, timing.
3. Agree on **scope, deliverables, timeline and roles**.
4. Spot **problems early**: confounding, pseudo-replication, missing controls, a sample that is too small, ethical or data-access issues.

It is not the place to show off methods. Most of your time should be spent listening and asking questions.

## A question list that works

**The question**
- What decision will this analysis inform? What would you do differently if the answer were yes versus no?
- What result would surprise you? What do you already believe?
- Is the goal to estimate an effect, compare options, predict, or explain?

**The design and data**
- What is the experimental (or observational) unit? How many are there?
- How were treatments assigned? Was there randomisation, blocking, blinding?
- Are there repeated measurements on the same unit? Batches, plates, days, sites?
- What exactly is measured, with what instrument, and what is its precision? Are there detection limits?
- What is missing, and why?

**The constraints**
- Deadline, budget, number of runs still possible.
- Regulatory or reporting requirements (a GMP environment, a clinical protocol, a journal's guidelines).
- Data access, privacy and who owns the data.

**The deliverable**
- Who is the audience: the scientist, a manager, a regulator, a journal?
- What form: a short memo, a figure, a full report, code they can rerun?

## Translating between languages

Collaborators speak their domain's language; you speak statistics. Restate the question in both, and check you agree:

```text
Collaborator: "Does the new buffer work better?"
Restated:     "Among assays run on the same day, is mean enzyme activity with
               buffer B higher than with buffer A, and by how much? A difference
               of at least 5 units would change which buffer we buy."
```

The restated version names the unit, the comparison, the blocking factor and the **smallest important difference**, which you need for power and for interpreting the result.

## Red flags to raise early

- "We measured each sample three times, so n = 30" (with 10 samples): pseudo-replication.
- "All the treated samples were run last week": treatment confounded with time.
- "We'll see what's significant": no pre-specified hypothesis, high risk of false positives.
- "Can you get p below 0.05?": raise it politely and firmly. Your job is a sound analysis, not a particular answer. The American Statistical Association's **Ethical Guidelines for Statistical Practice** (amstat.org) cover exactly this situation.

## Scope and expectations

End the intake with a short written summary, sent the same day:

- the question as agreed;
- the data you will receive, in what format, by when;
- the planned analysis (in plain words) and what it can and cannot conclude;
- deliverables, dates and authorship or acknowledgement expectations;
- open questions.

This one-page summary prevents most later disputes. Update it if the scope changes.

## Careers and credentials

Consulting skills are central for statisticians, data scientists, operations research analysts and lab scientists who become the "stats person" in their group. Formal requirements vary: a master's degree is typical for statistician roles, and the ASA offers an optional **PStat** accreditation for experienced practitioners. INFORMS offers the optional CAP for analytics professionals. Check the current requirements with the official bodies (amstat.org, informs.org) rather than relying on summaries. Checked on 2026-10-23.

## Common mistakes

- **Jumping to methods** before understanding the question.
- **Accepting the data as given** without asking how it was produced.
- **Promising a result** (or a p-value) instead of an analysis.
- **Not writing anything down**, so scope creeps and memories differ.
- **Using jargon** the collaborator cannot check. If they cannot explain your conclusion to their boss, it will be misused.

## Quick recap

- The intake is for listening: the question, the data-generating process, the constraints, the deliverable.
- Restate the question in measurable terms, including the smallest important difference.
- Raise design problems early and ethically; send a written scope summary.

## Try it

The **Do** tab has you run a practice intake with a friend, classmate or colleague (or a role-play script), using the question list above, and write the one-page summary.


Run a statistical consulting intake that pins down the question, the data and the deliverable.

Working with researchers

My role as a statistical consultant

Statistical Consultants - Statistics Without Borders

Methods, design: describe the units (batches), the factor and levels, the number of replicates and how run order was randomised.

Methods, analysis: name the model and test (one-way ANOVA), the follow-up procedure (Tukey's HSD), the significance level and the software you would use, with a version number.

Methods, assumptions: say how you checked residuals and equal variances (group SDs here are about 2.2, 1.6 and 1.6).

Results: give the group means with the number of batches, the ANOVA result as F(df1, df2) with an exact p-value, and the eta-squared effect size (143.3 / 183.3 = 0.78).

Results: report the pairwise differences in yield points, with a 95% confidence interval for at least the B minus A difference (compute it from Tukey output in Python or R, or note that you would).

Add a three-sentence plain-language summary for a plant manager at the top: the answer, how sure you are, and a recommendation.

Check your draft against the lesson's common-mistakes list (effect sizes, exact p-values, no 'n.s.', sensible rounding, no causal overreach) and fix anything you find.

## Why this matters

An analysis that nobody can follow, check or reproduce is not finished. Whether the audience is a journal, a regulator, a client or your own team in six months, the **methods** and **results** sections are where your statistical work becomes evidence. Good statistical writing is plain, specific and complete. It tells the reader what was done in enough detail to repeat it, and what was found with enough context (effect sizes, uncertainty) to judge it.

## The methods section: enough to reproduce

A reader should be able to redo your analysis from the methods. Cover, in order:

1. **Design.** Units, factors and levels, blocking, randomisation (how it was done), replication, blinding. Planned sample size and how it was chosen (the power calculation).
2. **Measurements.** The response(s), instrument, units, any transformations, how detection limits or missing values were handled.
3. **Statistical model.** Name it and write its terms: "a linear mixed model with buffer and day of week as fixed effects and plate as a random intercept". State the link and distribution for GLMs.
4. **Inference.** Tests and intervals used, the significance level, the multiple-comparison adjustment, one- or two-sided.
5. **Assumption checks** and what you did when they failed.
6. **Software and versions**, with the key packages: "Python 3.12, statsmodels 0.14". Point to code and data if they can be shared.
7. **Deviations from the plan** (excluded runs and why, a changed model).

Example paragraph:

```text
Four buffers were compared in a randomised complete block design with day
(n = 4) as the block; within each day, the run order of the buffers was
randomised by computer (seed recorded). Enzyme activity (U/mL) was analysed
with a linear model containing buffer and day as fixed effects. Pairwise
differences between buffers were assessed with Tukey's HSD at a family-wise
alpha of 0.05. Residual plots showed no departures from constant variance or
normality. Analyses used Python 3.12 with statsmodels 0.14; code is available
in the project repository.
```

## The results section: estimates first

Lead with **what you found and how big it is**, then the uncertainty, then the test. Not "the result was significant (p < 0.05)", but:

```text
Mean activity with buffer B was 6.2 U/mL higher than with buffer A
(95% CI 3.1 to 9.3; Tukey-adjusted p = 0.002). Buffers A, C and D did
not differ by more than 2.1 U/mL (all adjusted p > 0.4).
```

Rules of thumb:

- **Report effect sizes with confidence intervals**, in the units of the response.
- **Give exact p-values** (p = 0.03, p = 0.21), with `p < 0.001` as the floor. Never just "n.s.".
- **Report sample sizes** for every comparison, and say if any data were excluded.
- **Use tables** for many numbers (an ANOVA table, a coefficient table) and **figures** for patterns (interaction plots, residual plots, response surfaces). Do not repeat the same numbers in both.
- **Round sensibly:** two or three significant figures for effects, matching the measurement precision.
- **Keep interpretation separate.** The results say what the data show; the discussion says what it means.

## Writing for non-statisticians

Many reports go to managers or clients. Add a short **summary** at the top in plain English: the question, the answer, how sure you are, and what you recommend. Then the technical detail. For example: "Buffer B gives about 6 U/mL more activity than our current buffer, roughly a 10% improvement. The experiment was designed to detect differences of 5 U/mL, so this result is reliable. We recommend switching to B and confirming over the next month's production."

## Reporting guidelines

Many fields have checklists that reviewers expect:

- **CONSORT** for randomised clinical trials, **STROBE** for observational studies, **ARRIVE** for animal research.
- **SAMPL** (Statistical Analyses and Methods in the Published Literature) for general statistical reporting.
- The **EQUATOR Network** (equator-network.org) indexes these and many more.

Check your target journal's or agency's current requirements; they change. Checked on 2026-10-23.

## Common mistakes

- **"Statistically significant" without an effect size**, or "no effect" when the interval is wide.
- **Unnamed tests** ("a t-test was used") with no indication of paired or unpaired, equal or unequal variances.
- **Missing software versions and seeds**, which make reproduction impossible.
- **Selective reporting**: only the analyses that "worked". Report all planned analyses and label exploratory ones as exploratory.
- **Too many decimal places** (p = 0.0432187) and inconsistent rounding.
- **Causal language for observational results** without the assumptions from the causal inference lesson.

## Quick recap

- Methods: design, measurements, model, inference, assumption checks, software, deviations.
- Results: estimate, interval, then p-value; exact p-values and sample sizes.
- Add a plain-language summary for non-specialists; follow your field's reporting guideline.

## Try it

In the **Do** tab you will write a methods and results section for the catalyst ANOVA from module 2 and check it against a short checklist.


Write a reproducible methods section and a results section that leads with effect sizes and uncertainty.

Statistical reports

How do I write a statistical analysis and results section?

Create a project folder with `data/raw/`, `src/` (or `R/`) and a `README.md`. Save the catalyst data (15 yields and their catalyst labels) as `data/raw/catalyst.csv` and do not edit it again.

In Python: read the CSV with pandas and run `scipy.stats.f_oneway` and `statsmodels` `anova_lm`. In R: read it with `read.csv` and run `summary(aov(y ~ catalyst, data = d))`.

Confirm you get F = 21.5 on (2, 12) degrees of freedom and p of about 0.0001 in both tools. If anything differs, find the default that explains it.

Add Tukey's HSD (`pairwise_tukeyhsd` in Python, `TukeyHSD` in R) and check that only B differs from the others.

Record the environment: Python `pip freeze > requirements.txt` (or the version of each package used), R `sessionInfo()` or `renv::snapshot()`.

Write three lines in the README explaining how to rerun the analysis from scratch.

Restart the session (or 'Restart and run all' in a notebook) and rerun everything to prove it works from a clean state.

Optional: put the folder under Git and make a first commit with a message describing the analysis.

## Why this matters

A statistical result is only as trustworthy as your ability to produce it again. Reviewers, auditors and future colleagues will ask for it, and so will you, six months later, when new data arrive or a number in the report looks odd. Reproducibility is also how errors get caught: a famous 2013 re-analysis of an influential economics paper found a spreadsheet error simply because the original authors shared their file. This lesson covers the main statistical software tools and the habits that make an analysis rerunnable.

## The main tools

| Tool | Strengths | Typical users |
|---|---|---|
| **R** | Widest range of statistical methods; `lme4`, `emmeans`, `ggplot2`, the tidyverse; R Markdown / Quarto | Statisticians, biostatistics, academic research |
| **Python** | `pandas`, `statsmodels`, `scipy`, `scikit-learn`, `matplotlib`; one language from data cleaning to production | Data science, ML, engineering, OR |
| **SAS** | Long history in pharma and government; validated procedures | Clinical trials, regulated industries |
| **JMP, Minitab, Design-Expert** | Point-and-click DOE and quality tools | Process engineers, Six Sigma, labs |
| **Stata, SPSS** | Survey and social-science analyses | Economics, public health, social sciences |

You do not need all of them. Be fluent in one of R or Python, able to read the other, and comfortable reading output from the point-and-click tools your collaborators use. Most methods in this course exist in both:

| Task | R | Python |
|---|---|---|
| Linear model | `lm(y ~ x1 + x2)` | `smf.ols("y ~ x1 + x2", df).fit()` |
| GLM | `glm(y ~ x, family = poisson)` | `smf.glm("y ~ x", df, family=sm.families.Poisson()).fit()` |
| ANOVA table | `anova(model)` / `car::Anova` | `anova_lm(model, typ=2)` |
| Mixed model | `lme4::lmer(y ~ x + (1 \| g))` | `smf.mixedlm("y ~ x", df, groups=df["g"]).fit()` |
| Tukey HSD | `TukeyHSD(aov(...))` | `pairwise_tukeyhsd(y, group)` |

**Defaults differ.** R's `anova()` gives sequential (Type I) sums of squares; statsmodels' `anova_lm(typ=2)` gives Type II. For unbalanced data these can disagree. Contrast coding, optimisers and degrees-of-freedom methods for mixed models also differ. When two tools give different answers, it is usually a default, and you should be able to find which one.

## Habits of a reproducible analysis

1. **Raw data is read-only.** Never edit the original file. All cleaning happens in code, so every change is recorded.
2. **One script (or notebook) runs everything**, from raw data to every number and figure in the report. If a number in the report was typed by hand, it can drift from the analysis.
3. **Fix random seeds** for randomisation, simulation, bootstrapping and train/test splits, and record them.
4. **Record versions.** In Python, `pip freeze > requirements.txt` or a `pyproject.toml`; in R, `renv::snapshot()`. Note the language version too.
5. **Use version control** (Git) for code and the report. Commit messages explain why things changed.
6. **Literate documents** (Quarto, R Markdown, Jupyter) mix code, output and text so the report regenerates from data. Restart and run all before sharing a notebook; out-of-order cells are a common source of irreproducible results.
7. **Organise the project** predictably:

```text
project/
  data/raw/          original files, never edited
  data/processed/    written by code only
  src/ or R/         analysis code
  reports/           Quarto / notebook that builds the report
  requirements.txt   or renv.lock
  README.md          how to rerun, in three lines
```

## Cross-checking in a second tool

A strong check on an important result is to rerun it independently: a colleague reproduces it from the methods section alone, or you reproduce it in a second tool. For the catalyst ANOVA from module 2:

```python
from scipy import stats
print(stats.f_oneway([82, 85, 88, 84, 86], [90, 92, 89, 93, 91], [84, 83, 86, 85, 82]))
# F_onewayResult(statistic=21.5, pvalue=0.000108...)
```

```r
d <- data.frame(y = c(82,85,88,84,86, 90,92,89,93,91, 84,83,86,85,82),
                catalyst = rep(c("A","B","C"), each = 5))
summary(aov(y ~ catalyst, data = d))   # F value 21.5
```

Free ways to try both without installing anything: **Google Colab** for Python and **webR** or **Posit Cloud** (free tier) for R. Free tiers and their limits change; check the provider's page before relying on one.

## Common mistakes

- **Copying numbers by hand** from output into a report.
- **"It works on my machine"**: no record of package versions.
- **Unseeded randomness**, so the bootstrap interval changes every run.
- **Hidden state in notebooks** (cells run out of order, variables defined in deleted cells).
- **Overwriting raw data** while cleaning.
- **Trusting one tool's default** without knowing what it is.

## Quick recap

- Know R or Python well and read the other; know how their defaults differ.
- Raw data read-only, one script end to end, seeds, versions, Git, literate reports.
- Cross-check important results in a second tool or by a colleague.

## Try it

In the **Do** tab you will rerun an analysis from this course in a second tool (R or Python), record the environment and confirm the numbers match.


Compare R and Python for statistics and rerun an analysis reproducibly in a second tool.

Statistical software and reproducibility

R vs Python: Which should you learn for reproducible data science (CC168)

Reproducible Research - Data Science: An Introduction - 6.4

Run a consulting intake with researchers, write clear methods and results sections, and keep analyses reproducible across R and Python.