Reproducible analysis

Why this matters

An analysis that only works on your laptop, in your head, on the day you ran it, is fragile. Next month the data updates and you cannot remember which cells you ran. A colleague cannot check your numbers. An auditor or regulator asks how a reserve figure was calculated and nobody can say. Reproducible analysisWork that someone else, or future you, can rerun from the raw data and get the same results. means someone else, or future you, can rerun the work from the raw dataThe original files exactly as downloaded or delivered, which are never edited by hand. and get the same results. In actuarial work and regulated industries this is often required; everywhere else it is simply good practice.

Reproducible versus replicable

• Reproducible: same data, same code, same results.
• ReplicableReaching consistent conclusions with new data and the same method.: new data, same method, consistent conclusions.

This lesson is about the first. It is the foundation for the second.

The core habits

1. Never edit raw data. Keep the original files exactly as downloaded or delivered. Every cleaning step happens in code, and the output is saved as a new file. Anyone can then see and rerun every change.

2. Use a clear folder structure.

claims-analysis/
  README.md          what this is, where the data came from, how to run it
  requirements.txt   software and versions
  data/
    raw/             original files, never edited
    processed/       outputs of cleaning scripts
  src/               scripts or notebooks, numbered in run order
  output/            tables and charts for the report

3. Script everything. Avoid manual copy-paste steps between tools. If a step must be manual, write it down in the READMEA file that explains the question, data sources, how to run the analysis and where the outputs go..

4. Make notebooks run from top to bottom. Notebooks such as Jupyter are excellent for exploring, but cells run out of order can produce results that cannot be recreated. Before you share, restart the kernel and run all cells in order.

5. Set seeds for randomness. Simulations, train/test splits and random samples should use a fixed seed so results repeat.

6. Record versions. Library updates can change results. List the versions you used, for example in a requirements.txt file.

7. Use version controlA system such as Git that records every change to code with a message, so you can return to the exact code behind a report.. Git records every change to your code with a message explaining why. It also lets you go back to the exact code behind last quarter's report.

8. Write a README. Explain the question, data sources (with dates and licences), how to run the analysis and where the outputs go.

Worked example: turning a messy analysis around

Before: An analyst downloads a claims spreadsheet, deletes some rows that look wrong, adds a column by hand, copies a pivot table into a slide and emails it. A month later the manager asks for the same chart with new data. The analyst cannot remember which rows were deleted.

After:

1. The original file is saved in data/raw/claims_2026-09.csv.
2. src/01_clean.py removes rows with negative claim amounts (and logs how many), adds the new column and writes data/processed/claims_clean.csv.
3. src/02_summary.py builds the summary table and chart in output/.
4. The README says: "Run python src/01_clean.py then python src/02_summary.py."

Next month the analyst drops the new file in data/raw, updates one file name and reruns. The manager also asks why 37 rows were excluded; the cleaning log answers immediately.

Make it checkable

• Add simple sanity checksSimple coded tests, such as totals matching the source or no negative ages, that catch errors in each step. in code: totals match the source, no negative ages, row counts are as expected.
• Print or save a short log of what each step did.
• Ask a colleague to run your project from the README before you rely on it.

Privacy and data handling

Reproducibility does not mean publishing everything. Do not put personal, confidential or licensed data in public repositories. Share code and documentation, and describe how authorised people can obtain the data.

Common mistakes

• Editing raw files by hand. Changes become invisible and unrepeatable.
• Out-of-order notebooks. Always restart and run all before sharing.
• Hard-coded personal file paths such as C:/Users/me/Desktop. Use paths relative to the project folder.
• No record of versions or seeds.
• Committing secrets or private data to version control.

Quick recap

• Reproducible means same data plus same code gives the same results.
• Keep raw data untouched, script every step, use a standard folder layout, set seeds, record versions, use Git and write a README.
• Test by rerunning from a clean start, and have someone else try.

Try it

Open the Do tab and set up a small reproducible project on your own computer.