Plan, build, evaluate and document an end-to-end machine learning project for your portfolio.
Why this matters
Hiring managers for data science and ML engineering roles want evidence that you can take a vague problem to a working, honest result. A capstone project is that evidence. One complete, well-documented project beats five tutorial notebooks: it shows you can frame a problem, handle messy data, choose a sensible baseline, evaluate without fooling yourself, and explain what you found. This lesson gives you a plan; the Do tab walks you through it.
The core idea: a small problem, done end to end
Scope is the most common reason capstones fail. Pick a problem you can finish in roughly 20 to 40 hours. The end-to-end chain matters more than a fancy model:
question -> data -> baseline -> model -> evaluation -> error analysis -> write-up -> (optional) demo
Choosing a problem
Good capstones have:
- A decision attached. "Predict which support tickets need a specialist, so they skip the first queue" is better than "classify tickets".
- Public, legally usable data. Sources: Kaggle datasets, the UCI Machine Learning Repository, Hugging Face Datasets, government open-data portals (data.gov, data.gov.uk, Eurostat), or data you collect yourself. Read the licence, and avoid personal data you have no right to use.
- Enough rows for an honest test set: aim for at least a few thousand for tabular problems.
- A domain you care about, so you can judge whether results make sense.
Example scopes that fit this course:
| idea | type | techniques from this course |
|---|---|---|
| Predict bike-share demand per hour from weather and calendar | tabular regression | gradient boosting, time-based split |
| Route customer complaints to the right department | text classification | embeddings + logistic regression, TF-IDF baseline |
| Detect defective parts from product photos | image classification | transfer learning, augmentation |
| Answer questions over a public handbook | LLM + retrieval | RAG, evaluation set |
The project plan
- Frame it. Write a one-paragraph problem statement: who decides what, what the model predicts, and the metric that matches the decision. Define "good enough" up front.
- Get and understand the data. Explore distributions, missing values and label balance. Write down how labels were produced and any obvious biases.
- Split before you model. Random, grouped (by customer, patient or product) or time-based, depending on how the model will be used. Lock the test set away.
- Baselines first. Predict the mean or the majority class, then a simple model (logistic or linear regression, TF-IDF). Every later result is compared with these.
- Improve deliberately. One change per run, tracked with MLflow (module 4, lesson 1). Use cross-validation or a validation set for every choice.
- Evaluate once on the test set, with an uncertainty estimate (bootstrap interval) and a breakdown by relevant groups.
- Error analysis. Read 20–50 errors. Group them ("short messages", "night-time hours", "blurry photos"). This is often the most insightful part of the write-up.
- Write it up. A README with the problem, data, approach, results table, error analysis, limitations and how to reproduce the result, plus a model card (lesson 2).
- Optional demo. A small Streamlit or Gradio app, or a Hugging Face Space (free tier), lets a reviewer try the model in a minute.
What a reviewer looks for
- A clear problem statement and metric, tied to a decision.
- No leakage, an honest split, and the test set used once.
- Baselines and a results table that compares against them.
- Reproducibility: pinned dependencies (
requirements.txt), fixed seeds, a script or notebook that runs top to bottom, data download instructions. - Readable code: functions instead of one giant cell, sensible names, no secrets committed.
- Judgement: limitations stated plainly, and next steps that follow from the error analysis.
Repository layout that works
capstone/
README.md problem, results, how to run
MODEL_CARD.md
requirements.txt pinned versions
data/README.md where to download the data (do not commit large or licensed files)
notebooks/01_eda.ipynb
src/train.py python src/train.py reproduces the final model
src/evaluate.py
reports/figures/
Compute and cost
Everything in this plan runs on a laptop or in free Colab and Kaggle notebooks. If you choose a hosted LLM API or cloud resources, check the pricing first, set a spending limit or budget alert, prefer free tiers, and delete resources when you finish. Never commit API keys; use environment variables.
Common mistakes
- Scope creep: starting a second model family before the first one is evaluated.
- Leakage through random splits of time series, or through features computed on the full dataset.
- Only accuracy on imbalanced data.
- No baseline, so a 92% score means nothing.
- A notebook that only runs on your machine because of hidden state or absolute paths.
- Skipping the write-up, which is what reviewers actually read.
Quick recap
- Pick a small problem with a real decision, usable data and a clear metric.
- Split first, build baselines, improve one change at a time and track every run.
- Test once, analyse errors, and write a README plus model card that a reviewer can follow and reproduce.
Try it
Open the Do tab and work through the capstone checklist. Share the finished repository in the course forum for feedback.