Go from classic ML to the models used at work: neural networks, embeddings, CNNs, gradient boosting and LLM apps, plus evaluation, experiment tracking and a capstone.

Applied Machine Learning

## Why this matters

Linear and logistic regression draw straight boundaries. Many real problems need curved ones: the pixels of a handwritten digit, the words in a sentence, the interaction between two sensor readings. A **neural network** learns those curved boundaries by stacking simple layers. The same few ideas (layers, activations, a loss, backpropagation and gradient descent) power everything from a two-layer model on a spreadsheet to a large language model. In this lesson you build the smallest useful network by hand so the frameworks in the next lessons stop being magic.

## The core idea: layers of weighted sums

A single **neuron** takes inputs `x`, computes a weighted sum plus a bias, then applies a non-linear **activation** function:

```text
z = w1*x1 + w2*x2 + ... + b
a = activation(z)
```

A **layer** is many neurons side by side, which is one matrix multiplication. A two-layer network for binary classification looks like this:

```text
hidden:  h = tanh(X @ W1 + b1)        X is (n, d), W1 is (d, k), h is (n, k)
output:  p = sigmoid(h @ W2 + b2)     W2 is (k,), p is (n,), each p in (0, 1)
```

The hidden layer turns each row into `k` new features, and the output layer is just logistic regression on those learned features. That is the key insight: **the network learns its own features**.

Why the activation? Without it, two layers collapse into one: `(X @ W1) @ W2` equals `X @ (W1 @ W2)`, still a straight line. `tanh`, `sigmoid` and `ReLU` (`max(0, z)`) bend the space so the next layer can separate classes a line cannot.

## The loss and the backward pass

For binary labels we use **binary cross-entropy**, the same loss as logistic regression:

```text
loss = -mean( y*log(p) + (1-y)*log(1-p) )
```

Training means nudging every weight against its gradient. **Backpropagation** is the chain rule applied layer by layer, from the output back to the input. For the network above the gradients are:

```text
dz2 = (p - y) / n                    error signal at the output, shape (n,)
dW2 = h.T @ dz2                      db2 = sum(dz2)
dh  = outer(dz2, W2) * (1 - h**2)    (1 - h**2) is the derivative of tanh
dW1 = X.T @ dh                       db1 = dh.sum(axis=0)
```

The `(p - y)` term is neat: with sigmoid plus cross-entropy, the derivative simplifies to "prediction minus truth". Each layer multiplies the incoming error by its local derivative and passes it back.

## Worked example: XOR

XOR is the classic problem a straight line cannot solve. The label is 1 when two inputs have the same sign:

| x0 | x1 | label |
|---|---|---|
| 0.8 | 0.6 | 1 |
| -0.7 | -0.4 | 1 |
| 0.5 | -0.9 | 0 |
| -0.6 | 0.3 | 0 |

Logistic regression tops out near 50% accuracy here. A network with 8 hidden `tanh` units learns to combine several tilted lines into a checkerboard boundary and reaches above 95%. A full training loop in NumPy is short:

```python
for epoch in range(3000):
    h, p = forward(X, params)          # forward pass
    grads = backward(X, y, h, p, params)
    for k in params:                   # gradient descent step
        params[k] -= lr * grads[k]
```

## Checking your gradients

Backprop bugs are silent: the code runs and the loss just falls slowly or not at all. The standard defence is a **numerical gradient check**. Nudge one weight by a tiny `eps` up and down, measure the loss each time, and compare:

```text
numerical = (loss(w + eps) - loss(w - eps)) / (2 * eps)
```

If that matches your analytic gradient to about six decimal places, your backward pass is right. Frameworks such as PyTorch compute gradients automatically (autograd), so you rarely do this at work, but it is the best way to understand what autograd does.

## Initialisation matters

If every weight starts at zero, every hidden unit computes the same thing and receives the same gradient, so they never become different. This is the **symmetry problem**. Networks start from small random weights, usually scaled by the layer size (Xavier/Glorot or He initialisation). Fix the random seed while you learn so your results are reproducible.

## Common mistakes

- **Forgetting the activation.** Stacked linear layers are still linear.
- **Shape errors.** Write the shape of every array in a comment, as above. Most bugs are a missing transpose.
- **Learning rate too high.** The loss jumps around or becomes `nan`. Halve it until the loss falls smoothly.
- **Unscaled inputs.** Features on very different scales make training slow. Standardise inputs first (mean 0, standard deviation 1).
- **Judging by training accuracy alone.** A network can memorise training data. The next lesson covers validation curves.

## Quick recap

- A layer is a matrix multiplication plus bias followed by a non-linear activation.
- Hidden layers learn features; the output layer turns them into a prediction.
- Backpropagation is the chain rule, layer by layer, from the loss back to each weight.
- Gradient checks catch backprop bugs; random initialisation breaks symmetry.

## Try it

Open the **Do** tab. You will write the forward and backward passes of a two-layer network in NumPy, pass a numerical gradient check, and train it to solve XOR.


Build a two-layer neural network in NumPy, write its backward pass and train it to solve XOR.

Neural network basics

But what is a neural network? | Deep learning chapter 1

Backpropagation, intuitively | Deep Learning Chapter 3

## Why this matters

Getting a deep model to train is the easy part. Getting it to **generalise**, to work on data it has never seen, is the job. Every training run ends in one of three states: the model is too simple to fit even the training data (**underfitting**), it fits the training data but not new data (**overfitting**), or it is about right. Reading those states from loss curves, and knowing which lever to pull for each, is the most practical skill in applied deep learning.

## The core idea: compare training and validation loss

Keep a validation set the model never trains on (you met this split in Machine Learning Foundations). After each epoch, record both losses and look at two things:

1. **Is the training loss low enough?** Compare it with a target: a simple baseline, human-level error, or the error your use case can tolerate.
2. **Is there a gap?** Validation loss much higher than training loss means the model memorised details that do not carry over.

```text
train high,  val high     -> underfitting  (high bias)
train low,   val high     -> overfitting   (high variance)
train low,   val close    -> good fit
```

### Worked example

You fit polynomials to 15 noisy points from `sin(3x)`:

| model | train MSE | val MSE | diagnosis |
|---|---|---|---|
| degree 1 (a line) | 0.18 | 0.20 | underfitting: too simple |
| degree 12, no penalty | 0.01 | 13.4 | overfitting: huge gap |
| degree 12, L2 penalty (lam = 0.01) | 0.03 | 0.04 | good fit |

The degree-12 model has enough capacity to pass near every training point, including the noise, and swings wildly between them. The penalty keeps its weights small so the curve stays smooth. The noise in this data has a variance of 0.04, so no model can do much better than a validation MSE of 0.04. You will reproduce this table in the activity.

## Fixing underfitting

- **More capacity:** more layers or units, a richer model family.
- **Train longer**, or fix the learning rate (too low a rate looks like underfitting).
- **Better features** or less regularisation.

## Fixing overfitting

- **More data**, or **data augmentation** (flipped or cropped images, paraphrased text), which is often the most effective fix.
- **L2 regularisation (weight decay):** add `lam * sum(w**2)` to the loss. For a linear model this has a closed form, ridge regression:

```python
import numpy as np

def ridge_fit(X, y, lam):
    # minimises ||Xw - y||^2 + lam * ||w||^2
    return np.linalg.solve(X.T @ X + lam * np.eye(X.shape[1]), X.T @ y)
```

  Larger `lam` means smaller weights and a smoother model. In PyTorch you pass `weight_decay=1e-4` to the optimiser; in Keras you add `kernel_regularizer=keras.regularizers.l2(1e-4)` to a layer.
- **Dropout:** during training, randomly zero a fraction of hidden units (for example 20–50%) so the network cannot rely on any single unit. It is switched off at prediction time.
- **Early stopping:** stop when the validation loss stops improving, and keep the best weights.
- **Smaller model**, as a last resort.

## Early stopping in detail

Validation loss usually falls, flattens, then rises as the model starts to memorise. Early stopping watches it with a **patience** setting:

```text
epoch:     0     1     2     3     4     5     6     7
val loss: 0.90  0.60  0.45  0.40  0.38  0.39  0.41  0.40
                                   best   +1    +2    +3 -> stop (patience 3)
```

Epoch 4 is the best, and three epochs pass without beating it, so training stops and the epoch-4 weights are restored. Keras has `keras.callbacks.EarlyStopping(patience=3, restore_best_weights=True)`; in PyTorch you write the few lines yourself or use a library such as PyTorch Lightning.

## Other training signals

- **Loss is `nan` or explodes:** learning rate too high, or bad input values. Lower the rate, check for missing values, and try gradient clipping.
- **Loss flat from the start:** the learning rate is too low, a bug zeroes the gradients, or the labels are shuffled relative to the inputs. Try to overfit a tiny batch of 10 examples first: a healthy model should drive the training loss near zero. If it cannot, there is a bug.
- **Validation loss noisy:** the validation set is too small. Use more data or cross-validation.

## Common mistakes

- **Tuning on the test set.** Every decision (epochs, `lam`, architecture) uses validation data. Touch the test set once at the end.
- **Reading training accuracy only.** It always improves with more capacity.
- **Applying dropout at evaluation.** Call `model.eval()` in PyTorch before validating; Keras handles this for you.
- **Adding regularisation to an underfitting model.** It makes the problem worse.
- **Changing several things at once.** Change one lever per run and record it (module 4 covers experiment tracking).

## Quick recap

- Underfitting: training loss itself is too high. Add capacity, train longer, regularise less.
- Overfitting: big gap between training and validation loss. Add data, augmentation, weight decay, dropout or early stopping.
- Early stopping keeps the weights from the best validation epoch.
- Before a long run, check the model can overfit a tiny batch.

## Try it

Open the **Do** tab. You will write a diagnosis function, an early-stopping rule and ridge regression in NumPy, then show that an L2 penalty rescues an overfitting degree-12 model.


Read training and validation curves to diagnose over- and underfitting, and fix them with regularisation and early stopping.

Training deep models

Overfitting, Underfitting, and Bad Data Are Ruining Your Predictive Models

Regularization Part 1: Ridge (L2) Regression

## Why this matters

You have written a forward pass, a backward pass and an update loop by hand. At work nobody does that: you use a deep learning framework that computes gradients automatically, runs on GPUs and ships tested layers. The two you will meet most are **PyTorch** (dominant in research and increasingly in industry) and **Keras** (a high-level API that runs on TensorFlow, JAX or PyTorch). Their vocabulary is the same: tensors, models, losses, optimisers and a training loop. Learn it once and you can read either.

## Where to run this

Neither framework runs in this site's browser sandbox, so this lesson is reading and quiz only. To try the code, use a free notebook: **Google Colab** or **Kaggle Notebooks** both have PyTorch and Keras preinstalled and offer a free GPU with usage limits. Locally, `pip install torch` (see pytorch.org for the right command for your machine) or `pip install keras tensorflow` inside a virtual environment.

## Tensors

A **tensor** is an n-dimensional array, like a NumPy array, with two extras: it can live on a GPU, and it can record the operations applied to it so gradients can be computed.

```python
import torch

x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
print(x.shape, x.dtype)        # torch.Size([2, 2]) torch.float32
w = torch.randn(2, 1, requires_grad=True)
y = (x @ w).sum()
y.backward()                   # autograd fills w.grad
print(w.grad)                  # dy/dw: the column sums of x, [[4.], [6.]]

device = "cuda" if torch.cuda.is_available() else "cpu"
x = x.to(device)               # move data to the GPU when there is one
```

`requires_grad=True` tells PyTorch to track operations on `w`. Calling `.backward()` on a scalar runs backpropagation for you, which is exactly what you wrote by hand last lesson.

## Models

In PyTorch a model is a class that inherits from `nn.Module`. You declare layers in `__init__` and describe the computation in `forward`:

```python
from torch import nn

class TwoLayerNet(nn.Module):
    def __init__(self, n_in, n_hidden):
        super().__init__()
        self.hidden = nn.Linear(n_in, n_hidden)   # holds W1 and b1
        self.out = nn.Linear(n_hidden, 1)         # holds W2 and b2

    def forward(self, x):
        h = torch.tanh(self.hidden(x))
        return self.out(h).squeeze(-1)            # raw scores (logits)

model = TwoLayerNet(2, 8)
```

The same network in Keras is declarative:

```python
import keras

model = keras.Sequential([
    keras.Input(shape=(2,)),
    keras.layers.Dense(8, activation="tanh"),
    keras.layers.Dense(1, activation="sigmoid"),
])
```

## Loss and optimiser

The **loss function** measures error; the **optimiser** updates the weights from the gradients. `SGD` is plain gradient descent; **Adam** adapts a step size per weight and is the usual default.

```python
loss_fn = nn.BCEWithLogitsLoss()     # sigmoid + cross-entropy, numerically stable
optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, weight_decay=1e-4)
```

Note that the PyTorch model returns **logits** (raw scores) and the loss applies the sigmoid. Combining them avoids overflow, so do not add your own sigmoid before `BCEWithLogitsLoss`.

## The training loop

PyTorch makes you write the loop, which is a little more code and a lot more control:

```python
from torch.utils.data import DataLoader, TensorDataset

loader = DataLoader(TensorDataset(X_train, y_train), batch_size=32, shuffle=True)

for epoch in range(20):
    model.train()                     # enables dropout and similar layers
    for xb, yb in loader:
        optimizer.zero_grad()         # clear old gradients
        loss = loss_fn(model(xb), yb) # forward pass
        loss.backward()               # backward pass
        optimizer.step()              # update weights

    model.eval()                      # disables dropout
    with torch.no_grad():             # no gradient tracking during evaluation
        val_loss = loss_fn(model(X_val), y_val).item()
    print(epoch, round(val_loss, 4))
```

The five lines inside the batch loop are the whole of training: zero the gradients, forward, compute the loss, backward, step. Keras wraps all of it in two calls:

```python
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
history = model.fit(X_train, y_train, validation_data=(X_val, y_val),
                    epochs=20, batch_size=32)
```

`history.history` holds the per-epoch training and validation loss, ready for the over- and underfitting diagnosis from the previous lesson.

## Saving and loading

```python
torch.save(model.state_dict(), "model.pt")     # PyTorch: save the weights
model.load_state_dict(torch.load("model.pt"))

model.save("model.keras")                       # Keras: architecture + weights
```

## Choosing between them

- **PyTorch:** explicit loops, easy debugging with ordinary Python, the largest ecosystem of research code and pretrained models (Hugging Face is PyTorch-first).
- **Keras:** less code for standard models, quick to prototype, built-in callbacks such as `EarlyStopping`.
- Both export models for serving (TorchScript or ONNX for PyTorch; SavedModel or ONNX for TensorFlow/Keras). Pick the one your team uses.

## Common mistakes

- **Forgetting `optimizer.zero_grad()`.** PyTorch adds new gradients to old ones, so steps get larger and wrong.
- **Forgetting `model.eval()` and `torch.no_grad()`** when validating, which leaves dropout on and wastes memory.
- **Device mismatch.** The model and every batch must be on the same device: `RuntimeError: Expected all tensors to be on the same device`.
- **Wrong dtype or shape for labels.** `BCEWithLogitsLoss` wants float labels with the same shape as the logits.
- **Double sigmoid.** A sigmoid output layer plus `BCEWithLogitsLoss` applies it twice.

## Quick recap

- Tensors are GPU-capable arrays that can track gradients; `.backward()` runs autograd.
- A model holds layers and defines `forward`; the optimiser (usually Adam) updates its parameters.
- One training step: zero gradients, forward, loss, backward, step. Validate with `eval()` and `no_grad()`.
- Keras `compile` + `fit` wraps the same loop.


Learn how PyTorch and Keras express tensors, models, losses, optimisers and the training loop.

PyTorch or Keras tour

PyTorch in 100 Seconds

PyTorch Tutorial 02 - Tensor Basics

Build and train a small neural network by hand, diagnose over- and underfitting, and learn how PyTorch and Keras express the same ideas.