Take derivatives of functions of several variables, build the gradient, and use it for gradient descent.
Why this matters
Most quantities depend on more than one input. The stress in a beam depends on load and length, the yield of a reaction on temperature and pressure, an insurance price on age and coverage, and a machine learning model's error on thousands of weights. To ask "how does the output respond if I change one input?", or "which way should I move all the inputs to improve the output fastest?", you need partial derivatives and the gradient.
Partial derivatives
A partial derivative measures the rate of change with respect to one variable while holding all the others fixed. It is written with a curly ∂:
f(x, y) = x²y + 3y
∂f/∂x = 2xy (treat y as a constant)
∂f/∂y = x² + 3 (treat x as a constant)
Every rule from Module 1 still applies; you just pretend the other variables are numbers. At the point (1, 2): ∂f/∂x = 4 and ∂f/∂y = 4.
Worked example from physics: the ideal gas pressure is P = nRT / V. Holding n and V fixed, ∂P/∂T = nR / V (pressure rises with temperature). Holding T fixed, ∂P/∂V = −nRT / V² (pressure falls as volume grows).
The gradient
Collect all the partial derivatives into a vector and you have the gradient:
∇f = (∂f/∂x, ∂f/∂y, …)
For f(x, y) = x²y + 3y at (1, 2), ∇f = (4, 4). The gradient has two key properties:
- It points in the direction of steepest increase of f.
- Its length is the rate of increase in that direction.
Picture a hilly landscape where f is the height. Standing anywhere, the gradient is an arrow pointing straight uphill. At the top of a hill, or the bottom of a valley, the ground is flat and the gradient is the zero vector, which is the multivariable version of f′(x) = 0.
Gradient descent
To minimise a function of many variables, repeatedly step against the gradient:
new point = point − learning_rate × ∇f(point)
This is gradient descent, the algorithm that trains linear regression, logistic regression and neural networks. Example: f(x, y) = (x − 3)² + (y + 1)² has ∇f = (2(x − 3), 2(y + 1)). Starting at (0, 0) with a learning rate of 0.1, the first step goes to (0, 0) − 0.1·(−6, 2) = (0.6, −0.2), heading toward the minimum at (3, −1). Too large a learning rate overshoots and can diverge; too small is slow.
Computing gradients numerically
When you only have a function, not a formula, estimate each partial with a central difference: nudge one coordinate up and down by h and leave the rest alone.
def partial(f, point, i, h=1e-5):
up = list(point)
down = list(point)
up[i] += h
down[i] -= h
return (f(up) - f(down)) / (2 * h)
def gradient(f, point):
return [partial(f, point, i) for i in range(len(point))]
f = lambda p: p[0] ** 2 * p[1] + 3 * p[1]
print(gradient(f, [1, 2])) # about [4.0, 4.0]
Machine learning frameworks (PyTorch, JAX, TensorFlow) instead use automatic differentiation, which applies the chain rule exactly through the code. Numerical gradients are still the standard way to test that an analytic or automatic gradient is correct ("gradient checking").
Common mistakes
- Changing two variables at once when estimating a partial. Only the i-th coordinate moves.
- Forgetting that other variables are constants. ∂/∂x of x²y is 2xy, not 2x.
- Stepping up the gradient when minimising. Descent subtracts the gradient.
- Mutating the input point in code. Copy it before nudging, or every later partial is wrong.
Quick recap
- A partial derivative holds every other variable fixed.
- The gradient ∇f stacks the partials; it points uphill, and its length is the steepness.
- At a minimum or maximum, ∇f = 0.
- Gradient descent steps against the gradient; it is how most models are trained.
Try it
In the Do tab, write the numerical partial derivative and use it to compute gradients and run a few steps of gradient descent.