Optimisation with derivatives

Why this matters

Engineers minimise material and weight, chemists maximise yield, operations researchers minimise cost, and machine learning minimises error. All of these are optimisation problems, and the derivative is the first tool for solving them: at the top of a hill or the bottom of a valley, the slope is zero.

The core idea

If f is smooth, a maximum or minimum inside an interval happens where f′(x) = 0. Those points are called critical points. To find the best value:

1. Write the quantity you care about as a function of one variable.
2. Differentiate it.
3. Solve f′(x) = 0 for the critical points.
4. Check each one, and the endpoints of the allowed interval, to see which gives the largest or smallest value.

To tell a maximum from a minimum, look at the sign of f′ on either side (positive then negative means a peak) or use the second derivative: f″(x) < 0 means the curve bends downward (a maximum), f″(x) > 0 means it bends upward (a minimum).

Worked example: the open box

You have a 30 cm × 30 cm sheet of metal. Cut a square of side x from each corner and fold up the sides to make an open box. What x gives the largest volume?

The base is (30 − 2x) on each side and the height is x, so:

V(x) = x(30 − 2x)²        for 0 < x < 15

Expand and differentiate:

V(x)  = 4x³ − 120x² + 900x
V′(x) = 12x² − 240x + 900 = 12(x² − 20x + 75) = 12(x − 5)(x − 15)

V′(x) = 0 at x = 5 and x = 15. At x = 15 the base has zero width (V = 0), so the candidate is x = 5. Check the second derivative: V″(x) = 24x − 240, and V″(5) = −120 < 0, so x = 5 is a maximum. The best box has volume 5 · 20² = 2,000 cm³.

When you cannot solve f′(x) = 0 by hand

Real models are rarely this tidy. Numerically, you can search for the point where the slope changes sign. Bisection is the simplest reliable method: if the slope is positive at the left end of an interval and negative at the right end, a peak lies between them. Check the midpoint; if the slope there is still positive, the peak is to the right, so move the left end up; otherwise move the right end down. Each step halves the interval.

def slope(f, x, h=1e-6):
    return (f(x + h) - f(x - h)) / (2 * h)

lo, hi = 0.01, 7.5
for _ in range(60):
    mid = (lo + hi) / 2
    if slope(volume, mid) > 0:
        lo = mid
    else:
        hi = mid

After 60 halvings the interval is far smaller than anything you could measure. Library optimisers such as scipy.optimize.minimize_scalar use smarter versions of the same idea, and gradient descent (Module 4) extends it to many variables.

Related rates, briefly

The derivative also links rates that change together. If a spherical balloon's radius grows at 2 cm/s, its volume V = (4/3)πr³ grows at dV/dt = 4πr² · dr/dt. At r = 10 cm that is 4π · 100 · 2 ≈ 2,513 cm³/s. The chain rule does the work: differentiate the relationship with respect to time.

Common mistakes

• Forgetting the endpoints. On a closed interval, the best value can sit at an edge where f′ is not zero.
• Assuming every critical point is a maximum. Check the second derivative or the sign change. Some critical points (like x³ at 0) are neither.
• Ignoring the physical domain. A negative length or a cut wider than half the sheet is not a real answer.
• Optimising the wrong quantity. Write down exactly what you are maximising, with units, before differentiating.

Quick recap

• Maxima and minima of a smooth function occur where f′(x) = 0 or at the ends of the interval.
• The second derivative tells a peak (f″ < 0) from a valley (f″ > 0).
• Bisection on the sign of the slope finds an optimum numerically.
• Related rates use the chain rule to connect rates of change over time.

Try it

In the Do tab, finish the bisection step that finds the best box size numerically.