Take models from notebook to production: reproducible training, serving APIs, retrainable pipelines, CI/CD for ML, drift detection, monitoring and governance.

MLOps: Production Machine Learning

Create a new folder, run `git init`, create a virtual environment (`python -m venv .venv`) and install only what you need (for example numpy, pandas, scikit-learn, mlflow).

Move your training code out of any notebook into `train.py`, with hyperparameters and the data path in a `params.json` file.

Pin the environment: run `pip freeze > requirements.txt` (or use a lock-file tool) and commit it together with the code.

Add data pinning: compute the SHA-256 of the data file in `train.py` and stop with a clear error if it does not match the hash stored in `params.json`.

Fix every random seed you use (Python `random`, NumPy, and the library's `random_state`) from one `seed` value in `params.json`.

Log each run with MLflow: parameters, the git commit (`git rev-parse HEAD`), the data hash, the main metric and `requirements.txt` as an artefact. Start the UI with `mlflow ui` and open http://127.0.0.1:5000.

Train twice from a clean commit and compare the two runs in the MLflow UI: the metric should match exactly or within a tiny tolerance you can explain.

Optional: write a Dockerfile with a pinned Python base image, build it and run the training inside the container, then confirm the metric matches again.

Clean up: stop the MLflow UI, and remove any Docker images you no longer need with `docker image rm <name>` so they do not fill your disk.

## Why this matters

A model in production is a promise: "this artefact, trained on this data with this code, behaves like the one we evaluated." If you cannot rebuild it, you cannot debug it, audit it or safely improve it. A surprising number of production incidents start with a sentence like "it worked on my laptop" or "we are not sure which data the current model was trained on." Reproducible training turns a one-off notebook result into something a team can trust.

## The core idea: pin three things

Every trained model is a function of three inputs. Pin all three and you can rebuild the model.

1. **Code**: the exact commit of your training script, feature code and configuration.
2. **Data**: the exact snapshot of the training data (not "the customers table", but "the customers table as of 2026-10-01, hash `9f2c...`").
3. **Environment**: the Python version, every library version and, ideally, the operating system image.

Add a fourth, smaller item: **randomness**. Shuffling, weight initialisation and train/test splits all use random numbers, so fix the seeds.

## Pinning code

Commit everything that affects training, including configuration such as hyperparameters. Keep configuration in a file (`config.yaml` or `params.json`) rather than in notebook cells you edited by hand. When you train, record the commit hash:

```bash
git rev-parse HEAD
```

If the working tree has uncommitted changes, the hash alone does not describe the code. Many teams make the training script refuse to run on a "dirty" tree, or at least log a warning.

## Pinning data

Data is large and changes, so you usually do not commit it to Git directly. Common options:

- **Content hashes.** Compute a SHA-256 of the data file and store it with the model. If the hash changes, the data changed.
- **Data version control (DVC).** DVC stores small pointer files in Git and the real data in a cache or remote storage, so `git checkout` plus `dvc checkout` restores matching code and data.
- **Immutable snapshots.** Write each training set to a dated, read-only location (for example `s3://bucket/train/2026-10-01/`) and never overwrite it.

```python
import hashlib

def file_sha256(path):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for chunk in iter(lambda: f.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()
```

## Pinning the environment

`pip install pandas` gives you whatever version is current today. Six months later it gives you a different one. Pin exact versions:

```text
# requirements.txt
numpy==2.1.3
pandas==2.2.3
scikit-learn==1.5.2
```

`pip freeze > requirements.txt` captures what is installed; lock-file tools (pip-tools, Poetry, uv) do the same with dependency resolution. A **container image** goes further and pins the operating system and system libraries too:

```dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "train.py"]
```

Pin the base image as well (`python:3.12-slim` moves over time; a digest such as `python:3.12-slim@sha256:...` does not).

## Recording a run

Experiment trackers such as **MLflow** save the parameters, metrics, code version and artefacts of each run. MLflow runs locally with no account: `pip install mlflow`, log from your script, and browse runs with `mlflow ui` at `http://127.0.0.1:5000`.

```python
import mlflow

with mlflow.start_run():
    mlflow.log_params({"seed": 42, "max_depth": 6, "data_sha256": data_hash})
    mlflow.log_param("git_commit", commit)
    mlflow.log_metric("val_auc", auc)
    mlflow.log_artifact("requirements.txt")
```

## Worked example: same inputs, same model

You train a gradient-boosted model twice on Monday and get validation AUC 0.871 both times. On Thursday a colleague gets 0.864. Because each run logged the commit, data hash and package versions, the diff takes one minute: the commit and data match, but their environment had a newer scikit-learn whose default changed. Pinning the version restores 0.871. Without the records, that investigation could take days.

Exact bit-for-bit equality is not always possible (GPU kernels and multi-threading can add tiny non-determinism). The practical goal is: same inputs give the same model within a small, known tolerance, and you can explain any difference.

## Common mistakes

- **Training from a live table.** The data changes under you. Snapshot first, then train on the snapshot.
- **Unpinned dependencies.** "Latest" is not a version.
- **Seeds set in one place only.** Python's `random`, NumPy and your framework each have their own generator.
- **Hand-edited notebooks.** Cells run out of order cannot be replayed. Move training into a script.
- **Not saving the preprocessing.** The scaler, vocabulary or encoder is part of the model; version it with the model.

## Quick recap

- A model is a function of code, data, environment and seeds: pin all four.
- Use Git commits, data hashes or DVC, exact package versions and a container image.
- Log every run (parameters, metrics, commit, data hash) with a tracker such as MLflow.
- Aim for "same inputs, same model within tolerance" and be able to explain any drift.

## Try it

Open the **Do** tab and make a small training project fully reproducible on your own machine, then prove it by training twice.


Pin code, data, environment and random seeds so any model can be rebuilt and explained.

Reproducible training

Version Control for Data Science Explained in 5 Minutes (No Code!)

MLOps explained | Machine Learning Essentials

## Why this matters

A model that lives only in a notebook helps nobody. To be useful it has to answer requests from other software: a website asking "is this transaction fraud?", a mobile app asking for recommendations. The most common way to do that is to wrap the model in a small web API. Getting this layer right (input validation, versioning, error handling) matters as much as the model itself, because it is what other teams actually depend on.

## The core idea: a thin, strict wrapper

A model-serving API does four things on every request:

1. **Parse and validate** the input against a schema. Reject bad requests with a clear error instead of guessing.
2. **Transform** the input into the exact features the model was trained on, using the same preprocessing code.
3. **Predict** with a model that was loaded once at start-up, not on every request.
4. **Respond** with the prediction plus metadata such as the model version, so callers and logs know which model answered.

Keep business logic out of this layer. The API is a contract: "send me these fields, get back this shape."

## Worked example: FastAPI

FastAPI is a popular Python framework for this. It uses Pydantic models to validate requests automatically and returns HTTP 422 when a field is missing or has the wrong type.

```python
# app.py  (run locally: pip install fastapi uvicorn; uvicorn app:app --reload)
from fastapi import FastAPI
from pydantic import BaseModel, Field
import joblib

MODEL_VERSION = "churn-2026-10-01"
model = joblib.load("model.joblib")          # loaded once, at start-up

class Customer(BaseModel):
    tenure_months: float = Field(ge=0, le=600)
    monthly_spend: float = Field(ge=0)
    support_tickets: int = Field(ge=0)

app = FastAPI()

@app.get("/health")
def health():
    return {"status": "ok", "model_version": MODEL_VERSION}

@app.post("/predict")
def predict(c: Customer):
    x = [[c.tenure_months, c.monthly_spend, c.support_tickets]]
    p = float(model.predict_proba(x)[0, 1])
    return {"churn_probability": round(p, 4), "model_version": MODEL_VERSION}
```

A request and response look like this:

```bash
curl -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"tenure_months": 3, "monthly_spend": 80, "support_tickets": 4}'
# {"churn_probability": 0.7311, "model_version": "churn-2026-10-01"}
```

FastAPI also generates interactive documentation at `/docs`, which is a handy way to test by hand.

## Status codes that matter

| situation | status |
|---|---|
| valid request, prediction made | 200 |
| body is not valid JSON | 400 |
| JSON is valid but a field is missing, the wrong type or out of range | 422 |
| the model or a dependency failed | 500 (and log it) |

Never return a prediction for input you could not validate. A silently filled-in default (for example treating a missing `tenure_months` as 0) produces confident nonsense that nobody notices.

## Packaging for deployment

Put the API in a container image with pinned dependencies (as in the previous lesson), expose one port, and add the `/health` endpoint so the platform can check the service is alive. You can run the image locally with Docker (`docker build -t churn-api .` then `docker run -p 8000:8000 churn-api`). Managed platforms (cloud container services, serverless functions, or specialised model servers such as BentoML, TorchServe or KServe) run the same image at scale. If you try a cloud service, use its free tier, set a budget alert first, and delete the service when you finish: an idle endpoint can still bill by the hour.

## Performance basics

- **Load once.** Reading a model file on every request can add hundreds of milliseconds.
- **Batch when you can.** Many models predict 100 rows almost as fast as 1.
- **Measure latency percentiles** (p50, p95, p99), not just the average: the slowest requests are what users notice.

## Common mistakes

- **Training/serving mismatch.** The API rebuilds features with different code from training. Share one feature function between both.
- **No version in the response.** When something goes wrong you cannot tell which model answered.
- **Accepting anything.** Missing range checks let negative ages or 10-million-dollar invoices through.
- **Leaking internals.** Return a short error message, not a stack trace.

## Quick recap

- Validate, transform, predict, respond: keep the serving layer thin and strict.
- Load the model once and include the model version in every response.
- Use 422 for invalid input and never invent defaults for missing fields.
- Ship it as a container with a health check; watch costs on any cloud deployment.

## Try it

Open the **Do** tab and write the request handler for a small model API in plain Python: validation, prediction and the right status codes.


Wrap a trained model in a strict web API that validates input, returns versioned predictions and uses the right status codes.

Serving models

Deploy ML models with FastAPI, Docker, and Heroku | Tutorial

## Why this matters

"Deploy the model" can mean very different systems. A nightly job that scores ten million customers and writes the results to a table is nothing like an API that must answer a payment request in 50 milliseconds. Picking the wrong pattern either wastes money (a real-time service nobody needed) or fails users (yesterday's predictions for a decision that depends on the last five minutes). Choosing the inference pattern is one of the first design decisions in any ML system.

## The core idea: when is the prediction needed?

Ask two questions:

1. **When must the prediction exist?** Before anyone asks (precompute), or only at the moment of the request?
2. **What data does it need?** Data that is already known hours ahead, or data that only exists at request time (the basket contents, the text just typed, the current location)?

The answers point to one of three patterns.

### Batch (offline) inference

A scheduled job loads a large set of rows, scores them all and writes predictions to storage (a database table, a file, a cache). Applications then *look up* the stored prediction.

- **Good for:** churn scores, weekly demand forecasts, product recommendations refreshed nightly, risk scores for a monthly review.
- **Strengths:** simple, cheap per prediction, easy to retry, easy to inspect before anyone sees the results, can use big models and slow features.
- **Weaknesses:** predictions are stale between runs, and you compute scores for rows nobody will ever look at.

### Online (real-time) inference

A service keeps the model in memory and answers each request synchronously, usually through an API like the one in the previous lesson.

- **Good for:** fraud checks at checkout, search ranking, content moderation on upload, anything that depends on request-time data.
- **Strengths:** fresh, uses context that only exists now, scores only what is asked for.
- **Weaknesses:** strict latency budgets, the service must stay up around the clock, features must be available quickly at request time, and cost is driven by peak traffic.

### Streaming (near-real-time) inference

A consumer reads events from a stream (for example Kafka or a cloud queue), scores each event within seconds and writes the result onward. Nobody waits on the response, but results are much fresher than a nightly batch.

- **Good for:** sensor anomaly alerts, updating a user's recommendations after each click, flagging suspicious log-ins for review.

## Worked example: choosing for three products

| product need | pattern | why |
|---|---|---|
| Email 2 million customers a "you may like" list every Monday | batch | the list can be computed days ahead; latency does not matter |
| Approve or decline a card payment | online | decision depends on this transaction; the customer is waiting |
| Alert an engineer when a pump's vibration looks abnormal | streaming | events arrive continuously; a few seconds of delay is fine |

A useful hybrid is **precompute plus online adjustment**: batch-compute candidate recommendations overnight, then re-rank the top 50 online using what the user did in this session. You get most of the freshness for a fraction of the cost.

## Latency and cost, roughly

An online service has a **latency budget**, for example "p99 under 100 ms end to end." That budget covers network time, feature lookups and the model, so a model that takes 300 ms on a CPU is simply not an option there, however accurate it is. Batch jobs have a **throughput** goal instead ("score 10 million rows before 6 a.m."), so they can use larger models and cheaper, interruptible compute.

On cost: a batch job pays only while it runs. An online endpoint pays for every hour it is up, often with at least one instance always on, even at 3 a.m. with no traffic. Start with batch unless the product truly needs request-time data.

## Common mistakes

- **Real-time by default.** Building an always-on service for predictions that could have been computed last night.
- **Batch for request-time decisions.** Using yesterday's score when the deciding signal (the current basket, the new message) did not exist yesterday.
- **Ignoring feature availability.** A feature that takes a 20-second warehouse query cannot be used online, even if it helped offline accuracy.
- **No fallback.** If the online model times out, what does the product do? Define a safe default (a rule, a cached score or "send to manual review").

## Quick recap

- Batch: precompute on a schedule, look up later. Cheap and simple, but stale.
- Online: answer each request synchronously. Fresh, but bound by latency and always-on cost.
- Streaming: score events within seconds without a waiting caller.
- Decide by when the prediction is needed and what data it depends on; start simple.

## Try it

Open the **Do** tab and take the quiz to practise choosing a pattern for realistic product requirements.


Choose between batch, online and streaming inference from when a prediction is needed and what data it depends on.

Batch vs online inference

"Online" prediction vs "batch" prediction in machine learning

Make training reproducible, serve a model behind a validated API, and choose between batch, online and streaming inference.