Limits and rates of change

Why this matters

Almost every quantity a scientist or engineer cares about is a rate: how fast a reaction uses up a reagent, how quickly a beam bends as load increases, how fast a model's error falls during training, how a premium changes as age goes up. Calculus is the toolkit for rates, and it starts with one idea: the limitThe value a function or expression heads towards as its input approaches a point, whether or not it is defined there..

You do not need to be a math whiz to use it. In practice you will compute most rates numerically with a few lines of code, and the formal rules in the next lesson are shortcuts for what the code does by brute force.

Average rate of change

Suppose a tank's water level (in centimetres) after t minutes is h(t) = t² + 2t. Between t = 1 and t = 3:

• h(1) = 1 + 2 = 3
• h(3) = 9 + 6 = 15
• average rate = (15 − 3) / (3 − 1) = 6 cm per minute

That is the slope of the straight line (the secant lineThe straight line through two points on a curve; its slope is the average rate of change between them.) joining two points on the curve. The general formula is the difference quotientThe expression (f(x + h) − f(x)) / h, the average rate of change of f over [x, x + h].:

average rate on [x, x + h] = (f(x + h) − f(x)) / h

From average to instantaneous

What is the rate exactly at t = 1? A single instant has no width, so we cannot divide by zero. Instead we shrink the interval and watch what the average rate approaches:

The values head toward 4. We write this as a limit:

lim (h → 0) of (f(x + h) − f(x)) / h = 4    at x = 1

Read it as: "as h gets closer and closer to 0, the quotient gets as close as we like to 4". The limit is the slope of the tangent lineThe line that touches a curve at one point; its slope is the instantaneous rate of change there., the instantaneous rate of change. It has a name you will use for the rest of this course: the derivativeThe instantaneous rate of change f′(x): the limit of the difference quotient as h approaches 0., written f′(x).

What a limit really says

A limit describes where a function is heading, not necessarily its value at that point. A classic example:

g(x) = (x² − 1) / (x − 1)

At x = 1 the formula gives 0/0, which is undefined. But for every other x, g(x) = x + 1, so as x approaches 1, g(x) approaches 2. The limit is 2 even though g(1) does not exist. Difference quotients always look like 0/0 at h = 0; the limit is what rescues them.

A limit fails to exist when the function jumps (different values from the left and right) or blows up toward infinity. Smooth physical quantities, such as position, temperature and concentration, usually have well-behaved limits, which is why calculus works so well in science.

Estimating a rate numerically

Code can do the shrinking for you. This is how many simulation and optimisation libraries compute derivatives when no formula is available:

def f(x):
    return x ** 2 + 2 * x

def forward_difference(f, x, h=1e-6):
    return (f(x + h) - f(x)) / h

print(forward_difference(f, 1))   # about 4.000001

A more accurate version looks both ways from x, which cancels most of the error:

def central_difference(f, x, h=1e-5):
    return (f(x + h) - f(x - h)) / (2 * h)

Common mistakes

• Making h too small. Computers store numbers with about 16 significant digits. With h = 1e-15 the subtraction f(x + h) − f(x) loses almost all its digits and the answer becomes noise. Values between 1e-4 and 1e-8 are a safe range.
• Confusing average and instantaneous rates. "The car averaged 60 km/h" does not mean the speedometer read 60 at 3:15 pm.
• Plugging in first. Substituting h = 0 into a difference quotient gives 0/0. Simplify or estimate first, then take the limit.
• Forgetting units. A rate has units of output per input: cm per minute, dollars per year, error per training step.

Quick recap

• Average rate = (f(x + h) − f(x)) / h, the slope of a secant line.
• The instantaneous rate is the limit of that quotient as h → 0: the derivative.
• A limit is where a function is heading; the function need not be defined there.
• Numerically, a small h (about 1e-6) gives a good estimate; the central differenceThe estimate (f(x + h) − f(x − h)) / (2h), which looks both ways from x and is more accurate than a one-sided difference. is more accurate.

Try it

Open the Do tab and write the difference quotient that estimates a rate of change, then use it on a few functions.