Eigenvalues

Why this matters

Most vectors change direction when a matrix acts on them. A few special vectors do not: the matrix only stretches or shrinks them. Those are eigenvectorsNon-zero vectors that a matrix only stretches or shrinks, without changing their line., and the stretch factors are eigenvaluesThe stretch factors λ in A v = λ v.. They describe the natural behaviour of a system: the vibration modes of a bridge, the principal stresses in a material, the stable states of a Markov chain, the energy levels in quantum mechanics, and the principal components in a dataset (PCA). Google's original PageRank was an eigenvector.

The core idea

A non-zero vector v is an eigenvector of a square matrix A, with eigenvalue λ (lambda), when:

A v = λ v

Applying A to v gives a multiple of v: same line, scaled by λ. If λ = 2 the vector doubles, if λ = 0.5 it halves, if λ is negative it flips, and if λ = 1 it is left unchanged.

A worked example

A = | 2  1 |
    | 1  2 |

Try v = (1, 1): A v = (3, 3) = 3·(1, 1). So (1, 1) is an eigenvector with eigenvalue 3.

Try v = (1, −1): A v = (1, −1) = 1·(1, −1). An eigenvector with eigenvalue 1.

Try v = (1, 0): A v = (2, 1), which is not a multiple of (1, 0). Not an eigenvector.

Geometrically, A stretches the plane by 3 along the diagonal line y = x and leaves the line y = −x alone.

Finding eigenvalues by hand

A v = λ v means (A − λI) v = 0 has a non-zero solution, which happens only when the determinant is zero:

det(A − λI) = 0

For the example: (2 − λ)² − 1 = 0, so 2 − λ = ±1, giving λ = 3 or λ = 1. Two useful checks: the eigenvalues add up to the traceThe sum of a matrix's diagonal entries, which equals the sum of its eigenvalues. (sum of the diagonal, 2 + 2 = 4) and multiply to the determinant (4 − 1 = 3).

Finding eigenvalues numerically

For anything larger than 3 × 3, nobody solves the characteristic equation by hand. Two numerical approaches:

Power iterationA method that repeatedly multiplies a vector by A and rescales it, so it lines up with the dominant eigenvector. finds the eigenvalue with the largest size. Start with almost any vector, multiply by A, rescale to length 1, and repeat. Each multiplication amplifies the dominant eigenvector more than the others, so the vector lines up with it. The Rayleigh quotientThe value (v · A v) / (v · v), which reads off the eigenvalue for an eigenvector v. (v · A v) / (v · v) then reads off its eigenvalue.

import numpy as np

A = np.array([[2.0, 1.0], [1.0, 2.0]])
v = np.array([1.0, 0.0])
for _ in range(50):
    v = A @ v
    v = v / np.linalg.norm(v)
print(v @ A @ v)   # about 3.0

Library routines (np.linalg.eig, and np.linalg.eigh for symmetric matrices) compute all eigenvalues and eigenvectors at once with the QR algorithm:

values, vectors = np.linalg.eigh(A)
print(values)      # [1. 3.]

Each column of vectors is an eigenvector, scaled to length 1.

Where eigenvalues show up

• Stability: in a system x_(n+1) = A x_n, if every eigenvalue has size below 1, the system settles down; if any is above 1, it grows. Control engineers and population ecologists check exactly this.
• Vibrations: a structure's natural frequencies come from the eigenvalues of its stiffness and mass matrices. Resonance happens when forcing matches one.
• Principal stresses: the eigenvectors of the stress tensor are the directions with no shear.
• PCA: the eigenvectors of a covariance matrix are the directions of greatest variance in the data.

Common mistakes

• Allowing v = 0. The zero vector satisfies A·0 = λ·0 for every λ, so it never counts.
• Expecting unique eigenvectors. Any non-zero multiple of an eigenvector is also one. Software returns unit-length versions, and the sign may flip.
• Using eig for symmetric matrices. eigh is faster, more accurate and returns real, sorted values.
• Forgetting complex eigenvalues. A rotation matrix moves every real vector off its line, so its eigenvalues are complex.

Quick recap

• A v = λ v: an eigenvector keeps its direction; λ is its stretch factor.
• Eigenvalues solve det(A − λI) = 0; they sum to the trace and multiply to the determinant.
• Power iteration finds the dominant eigenvalue; np.linalg.eigh finds them all.
• They describe stability, vibration modes, principal stresses and PCA.

Try it

In the Do tab, finish power iteration and compare its answer with NumPy's.