The calculus and linear algebra behind science, engineering and machine learning: derivatives, integrals, vectors, matrices, gradients and eigenvalues, with Python exercises.

Calculus & Linear Algebra for STEM

## Why this matters

A single number cannot describe a force acting at an angle, a wind velocity, the colour of a pixel or a customer's purchase history. A **vector** can. Structural engineers add force vectors to check that a joint balances, physicists resolve velocities into components, and data scientists treat every row of a dataset as a vector so they can measure how similar two rows are. Linear algebra, the second half of this course, is built on vectors.

## What a vector is

A vector is an ordered list of numbers. Geometrically, in 2D or 3D it is an arrow with a length and a direction; in data work it can have hundreds of entries.

```text
v = (3, 4)          a 2D vector: 3 right, 4 up
F = (120, −40, 0)   a force in newtons along x, y, z
```

The numbers are the vector's **components**. An arrow's position does not matter, only its length and direction.

## Adding and scaling

- **Addition** works component by component: (3, 4) + (1, −2) = (4, 2). Geometrically, place the arrows tip to tail.
- **Scalar multiplication** stretches the vector: 2 · (3, 4) = (6, 8). A negative scalar flips its direction.

Worked example: two cables pull on a ring with forces (300, 400) N and (−100, 200) N. The resultant is (200, 600) N. A third force of (−200, −600) N would balance the ring, so the sum would be the zero vector. That is the basic check of statics.

## Length (magnitude)

By Pythagoras, the length of v = (v₁, v₂, …, vₙ) is:

```text
|v| = √(v₁² + v₂² + … + vₙ²)
```

|(3, 4)| = √(9 + 16) = 5. Dividing a vector by its length gives a **unit vector** (length 1) pointing the same way: (0.6, 0.8).

## The dot product

The dot product multiplies matching components and adds them up:

```text
u · v = u₁v₁ + u₂v₂ + … + uₙvₙ
(1, 2, 3) · (4, 5, 6) = 4 + 10 + 18 = 32
```

It turns two vectors into one number, and that number has a geometric meaning:

```text
u · v = |u| · |v| · cos θ
```

where θ is the angle between them. So:

- u · v > 0: they point roughly the same way (θ < 90°).
- u · v = 0: they are **perpendicular** (orthogonal).
- u · v < 0: they point roughly in opposite directions.

Rearranging gives the angle: cos θ = (u · v) / (|u| |v|). Also notice that v · v = |v|², so length comes from the dot product too.

## Where the dot product shows up

- **Work in physics:** W = F · d. Pushing at an angle only counts the component of force along the motion.
- **Projections:** the component of u along a unit vector e is u · e. Engineers use it to resolve loads along a member.
- **Similarity in data:** cosine similarity, (u · v) / (|u| |v|), compares documents, user profiles and embeddings in search and recommendation systems.
- **Weighted sums:** a linear model's prediction w₁x₁ + w₂x₂ + … is the dot product w · x.

## In code

Plain Python handles small vectors well:

```python
import math

def dot(u, v):
    return sum(a * b for a, b in zip(u, v))

def length(v):
    return math.sqrt(dot(v, v))

print(dot([1, 2, 3], [4, 5, 6]))   # 32
```

With NumPy, `numpy.array([1, 2, 3]) @ numpy.array([4, 5, 6])` or `numpy.dot` does the same thing, fast, for millions of entries.

## Common mistakes

- **Adding vectors of different sizes.** Both must have the same number of components.
- **Treating the dot product as a vector.** It is a single number. (The cross product, used for torque in 3D, is a vector.)
- **Forgetting to normalise** before reading cos θ; divide by both lengths.
- **Mixing units.** Do not add a force vector in newtons to one in kilonewtons.

## Quick recap

- A vector is an ordered list of components: an arrow with length and direction.
- Add component-wise; scale by multiplying every component.
- Length is √(v · v); a unit vector has length 1.
- u · v = Σ uᵢvᵢ = |u||v|cos θ: zero means perpendicular.

## Try it

In the **Do** tab, write the dot product and use it to find lengths and angles.


Add and scale vectors, measure their length, and use the dot product to find angles and test for perpendicularity.

Vectors

Vectors | Chapter 1, Essence of linear algebra

Dot products and duality | Chapter 9, Essence of linear algebra

## Why this matters

Real problems rarely have one unknown. A truss has a force in every member, an electrical network has a current in every branch, a chemical equation has a coefficient for every species, and a regression model has a weight for every feature. Each gives a **system of linear equations**, and matrices are how engineers, scientists and software solve them, often with thousands of unknowns at once.

## What a matrix is

A matrix is a rectangular grid of numbers with m rows and n columns (an m × n matrix):

```text
A = | 2   1 |
    | 1   3 |
```

A vector can be thought of as a matrix with one column.

## Matrix times vector

To multiply A by a vector x, take the dot product of each **row** of A with x:

```text
| 2  1 | | x |   | 2x + y  |
| 1  3 | | y | = | x + 3y  |
```

So the whole system

```text
2x +  y = 5
 x + 3y = 10
```

is one equation: **A x = b**, with b = (5, 10). The matrix holds the coefficients, x holds the unknowns, and b holds the right-hand sides.

## Matrix times matrix

To multiply A (m × n) by B (n × p), entry (i, j) of the result is row i of A dotted with column j of B. The inner sizes must match, and the result is m × p. Order matters: in general AB ≠ BA.

```text
| 1  2 | | 5  6 |   | 1·5 + 2·7   1·6 + 2·8 |   | 19  22 |
| 3  4 | | 7  8 | = | 3·5 + 4·7   3·6 + 4·8 | = | 43  50 |
```

## Solving A x = b

By hand, elimination works: subtract 2 × (second equation) from the first to get −5y = −15, so y = 3 and x = 1. Check: 2·1 + 3 = 5 and 1 + 3·3 = 10.

Conceptually, if A has an **inverse** A⁻¹ (the matrix that undoes A), then x = A⁻¹b. For a 2 × 2 matrix:

```text
A = | a  b |      A⁻¹ = 1/(ad − bc) · |  d  −b |
    | c  d |                          | −c   a |
```

The number ad − bc is the **determinant**. If it is 0 the matrix has no inverse: the equations are either contradictory or redundant, so there is no unique solution.

In practice you never compute an inverse to solve a system. You call a solver that uses elimination (LU decomposition), which is faster and more accurate:

```python
import numpy as np

A = np.array([[2.0, 1.0],
              [1.0, 3.0]])
b = np.array([5.0, 10.0])

x = np.linalg.solve(A, b)
print(x)                       # [1. 3.]
print(np.allclose(A @ x, b))   # True: always check the residual
```

`@` is Python's matrix multiplication operator. `np.linalg.det(A)` gives the determinant (here 5).

## A three-unknown example

A lab mixes three stock solutions to hit target amounts of three nutrients. Each column of A is one solution's nutrient content per litre, x is the litres of each, and b is the target. One call to `np.linalg.solve(A, b)` gives the recipe. If the answer contains a negative volume, the target is not reachable with those stocks, and the maths is telling you something real.

## Common mistakes

- **Mismatched sizes.** An m × n matrix times a vector needs n components. NumPy will raise an error, which is helpful.
- **Using `*` instead of `@`.** In NumPy, `A * B` multiplies element by element; `A @ B` is matrix multiplication.
- **Assuming AB = BA.** Matrix multiplication is not commutative.
- **Inverting to solve.** Use `np.linalg.solve`, not `np.linalg.inv(A) @ b`.
- **Ignoring a near-zero determinant.** A nearly singular system amplifies measurement errors; the answer may be meaningless.

## Quick recap

- A system of linear equations is A x = b: coefficients, unknowns, right-hand side.
- Matrix-vector and matrix-matrix products are built from dot products of rows and columns.
- A unique solution exists when det(A) ≠ 0.
- Solve with `np.linalg.solve` and check that A @ x is close to b.

## Try it

In the **Do** tab, turn a three-equation system into a NumPy matrix and solve it.


Write a system of linear equations as A x = b, multiply matrices, and solve the system with NumPy.

Matrices and systems

Matrices to solve a system of equations | Matrices | Precalculus | Khan Academy

Matrix multiplication as composition | Chapter 4, Essence of linear algebra

## Why this matters

So far a matrix has been a table of coefficients. There is a more powerful way to see it: a matrix is a **function that moves vectors**. It can rotate, stretch, shear or flatten space. Computer graphics and game engines rotate every vertex with a matrix, robotics chains rotations to place a gripper, materials scientists describe strain with a matrix, and data scientists reduce dimensions by projecting data. Once you see matrices as transformations, eigenvalues (next module) make sense.

## The core idea

A **linear transformation** T takes a vector in and gives a vector out, with two rules: T(u + v) = T(u) + T(v) and T(c·v) = c·T(v). Grid lines stay straight, parallel and evenly spaced, and the origin stays put. Every linear transformation of 2D space is multiplication by a 2 × 2 matrix, and the matrix is easy to read:

**The columns of the matrix are where the basis vectors land.**

The basis vectors are i = (1, 0) and j = (0, 1). If a transformation sends i to (a, c) and j to (b, d), its matrix is

```text
| a  b |
| c  d |
```

and any vector (x, y) = x·i + y·j lands at x·(a, c) + y·(b, d).

## A gallery of transformations

| Transformation | Matrix | Effect on (1, 0) and (0, 1) |
|---|---|---|
| Scale by 2 in x, 3 in y | [[2, 0], [0, 3]] | (2, 0), (0, 3) |
| Rotate 90° counter-clockwise | [[0, −1], [1, 0]] | (0, 1), (−1, 0) |
| Rotate by angle θ | [[cos θ, −sin θ], [sin θ, cos θ]] | (cos θ, sin θ), (−sin θ, cos θ) |
| Horizontal shear | [[1, k], [0, 1]] | (1, 0), (k, 1) |
| Reflect across the x-axis | [[1, 0], [0, −1]] | (1, 0), (0, −1) |
| Project onto the x-axis | [[1, 0], [0, 0]] | (1, 0), (0, 0) |

Worked example: rotate the point (2, 1) by 90°.

```text
| 0  −1 | | 2 |   | 0·2 − 1·1 |   | −1 |
| 1   0 | | 1 | = | 1·2 + 0·1 | = |  2 |
```

## Composition is multiplication

Applying B first and then A is the single transformation AB. Order matters: rotating then shearing is not the same as shearing then rotating, which is exactly why matrix multiplication is not commutative. Two rotations combine as you would hope: rotation(α) · rotation(β) = rotation(α + β).

## The determinant as area scale

The determinant of a 2 × 2 matrix tells you how much the transformation scales area. The unit square (area 1) becomes a parallelogram of area |det|:

- Scaling by 2 and 3: det = 6, areas grow six-fold.
- Any rotation: det = 1, areas unchanged.
- A shear: det = 1, the square leans but keeps its area.
- A projection: det = 0, space is flattened onto a line, which is why it has no inverse.
- A negative determinant means the transformation flips orientation (like a mirror).

## In code

```python
import numpy as np

def rotation(theta):
    c, s = np.cos(theta), np.sin(theta)
    return np.array([[c, -s], [s, c]])

R = rotation(np.pi / 2)
print(R @ np.array([2.0, 1.0]))    # about [-1, 2]
print(np.linalg.det(R))            # 1.0
```

To transform many points at once, stack them as columns of a matrix P and compute R @ P.

## Common mistakes

- **Putting the images in rows instead of columns.** The column convention is what makes A @ x work.
- **Angles in degrees.** `np.cos` expects radians; convert with `np.radians`.
- **Reading AB as "A first".** In AB applied to x, B acts first.
- **Forgetting translation is not linear.** Moving every point by a fixed offset moves the origin, so it is not a 2 × 2 matrix. Graphics uses 3 × 3 "homogeneous" matrices to include it.

## Quick recap

- A matrix is a linear transformation: its columns are the images of the basis vectors.
- Rotations, scalings, shears, reflections and projections all have simple matrices.
- Composing transformations multiplies their matrices, and order matters.
- The determinant is the area scale factor; 0 means space is flattened.

## Try it

In the **Do** tab, build a rotation matrix and test that it behaves like a rotation.


Read a matrix as a transformation of space: rotations, scalings and shears, composition, and the determinant as area scale.

Linear transformations

Linear transformations and matrices | Chapter 3, Essence of linear algebra

Vectors and dot products, matrices and linear systems, and matrices as transformations of space, with NumPy.