Clustering

Why this matters

Marketing teams want customer segments, biologists want groups of cells with similar gene expression, operations teams want to know which stores behave alike. None of these come with labels. Clustering finds groups of rows that are similar to each other and different from other groups. k-means is the simplest and most widely used clustering algorithm, and it is a good way to see how unsupervised learning works.

The k-means idea

You choose the number of clusters k. Each cluster is represented by a centroid, the average point of its members. k-means looks for centroids that make the total squared distance from each row to its own centroid as small as possible. That total is called inertia (or within-cluster sum of squares).

The algorithm

1. pick k rows at random as the starting centroids
2. repeat until nothing changes:
     assign: give each row to its nearest centroid
     update: move each centroid to the mean of its rows

Each assign step and each update step can only lower the inertia, so the algorithm always settles, usually within a few dozen iterations.

Worked example in one dimension

Points: 1, 2, 3, 10, 11, 12, k = 2, starting centroids 1 and 2.

assign: {1} -> centroid 1;  {2, 3, 10, 11, 12} -> centroid 2
update: centroids become 1 and (2+3+10+11+12)/5 = 7.6
assign: {1, 2, 3} closer to 1;  {10, 11, 12} closer to 7.6
update: centroids become 2 and 11
assign: nothing changes -> done

Two clusters, centred at 2 and 11, exactly as you would draw them by eye.

Vectorised assignment

With NumPy you compute all row-to-centroid distances at once by broadcasting:

d = ((X[:, None, :] - centroids[None, :, :]) ** 2).sum(axis=2)   # shape (n_rows, k)
labels = d.argmin(axis=1)

Scale your features first

k-means uses Euclidean distance, so a feature measured in large units dominates. If annual spend ranges over 0–120 (thousand dollars) and visits per month over 0–10, distances are almost entirely about spend. Standardise each column (subtract the mean, divide by the standard deviation) before clustering, then convert centroids back to original units to describe the segments.

Initialisation and restarts

Random starting centroids can land badly, for example two centroids inside one true group, and k-means then gets stuck in a poor solution (a local minimum). Two standard fixes:

• Restarts: run k-means several times with different seeds and keep the result with the lowest inertia.
• k-means++: choose starting centroids spread far apart. It is scikit-learn's default.

Choosing k

Inertia always falls as k grows (with k = n it is zero), so you cannot just minimise it. Common approaches:

• Elbow plot: plot inertia against k and look for the point where the curve bends and further clusters help little.
• Silhouette score (sklearn.metrics.silhouette_score): compares each row's distance to its own cluster with its distance to the next cluster; higher is better, from -1 to 1.
• Usefulness: can the business act on 4 segments? Would 12 be too many to name?

Interpreting segments

After clustering, describe each cluster with its centroid (in original units) and size, and give it a plain name:

segment   spend ($k)   visits/month   size   name
   A          20            2          60    occasional low spenders
   B          60            8          60    frequent regulars
   C         110            4          60    high-value planners

Then check that the segments are stable (similar on a new sample or another seed) and different on things you did not cluster on, such as churn rate. That is what makes them useful.

In scikit-learn

from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans

X = StandardScaler().fit_transform(raw)
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
print(km.cluster_centers_, km.inertia_)
labels = km.labels_

Limits of k-means

k-means assumes clusters that are roughly round and of similar size. It struggles with long, curved or very unequal clusters, and every point is forced into some cluster, including outliers. Alternatives: DBSCAN (density-based, finds arbitrary shapes and labels noise), Gaussian mixture models (soft, elliptical clusters) and hierarchical clustering (a tree of merges you can cut at any level).

Common mistakes

• Not scaling features, so one column decides everything.
• A single run with one seed, which can return a poor local minimum.
• Treating clusters as ground truth. They are a useful summary, not natural kinds.
• Clustering on IDs or dates encoded as numbers.

Quick recap

• k-means alternates assign (nearest centroid) and update (cluster mean) to reduce inertia.
• Standardise first, use several restarts or k-means++, and pick k with an elbow plot, silhouette score and judgement.
• Describe each segment by its centroid in original units and check it is stable and useful.

Try it

On the Do tab, write the assign and update steps and segment 180 customers into three groups.