Intro to system design

Why it matters

Writing a correct function is one skill. Deciding how the pieces of a service fit together, so that it keeps working when ten times more people use it, is another. System design is that second skill: choosing components (servers, databases, caches, queues), how they talk, and where the limits are. You will use it when you plan a new feature at work, read a design doc from a teammate, or answer a system design interview question. You do not need to be an expert in distributed systems to start. You need a repeatable process and a handful of building blocks.

A repeatable process

1. Clarify requirements. Write down what the system must do (functional requirements) and how well (non-functional: latency, availability, data size). "Users can shorten a URL and be redirected" is functional. "Redirects should take under 100 ms and the service should survive one server failing" is non-functional.
2. Estimate the load. Rough numbers decide the design. 1,000 new links a day fits on one small server; 100 million a day does not.
3. Define the API. What requests come in and what goes out.
4. Design the data model. What you store and how you look it up.
5. Draw the high-level design. Boxes and arrows: client, load balancer, app servers, database, cache.
6. Find the bottlenecks and fix the biggest one. Then repeat.

Worked example: a URL shortener

Requirements. Create a short link for a long URL; visiting the short link redirects to the long one. Links never expire. Reads (redirects) are far more common than writes.

Estimate. Say 1 million new links a day and 100 redirects per link over its life.

• Writes: 1,000,000 / 86,400 seconds is about 12 per second.
• Reads: 100 times that, about 1,200 per second on average; plan for peaks of maybe 5 times the average.
• Storage: about 500 bytes per link × 365 million links a year is roughly 180 GB a year. That fits comfortably in one managed database for years.

The numbers say: reads dominate, storage is modest, and one database is fine at first.

API.

POST /links        { "url": "https://example.com/very/long/path" }  -> 201 { "code": "aZ3kQ9" }
GET  /:code        -> 301 Location: https://example.com/very/long/path   (or 404)

Data model. One table, links(code PRIMARY KEY, url, created_at). The primary key makes lookups by code fast. The code can be a random 6–8 character string from 62 symbols (a–z, A–Z, 0–9): 62^7 is about 3.5 trillion combinations, so collisions are rare; on insert, retry if the code already exists.

High-level design.

client -> load balancer -> app servers (stateless, 2+) -> cache -> database

• Stateless app servers keep no user data in memory, so you can run two or more behind a load balancer and lose one without downtime. Adding servers is horizontal scaling; buying a bigger server is vertical scaling. Vertical is simpler until you hit the biggest machine or need redundancy.
• A cache (an in-memory key-value store such as Redis) holds popular code -> url pairs. Since a small share of links get most of the traffic, the cache absorbs most reads and the database sees a fraction.
• A database replica (a read-only copy) can serve reads if the cache is not enough, and takes over if the primary fails.

Bottlenecks. At 6,000 peak reads per second, the database would be the first thing to struggle without a cache. With a cache, the next questions are: what happens when the cache is empty after a restart (a "cold cache"), and how do you stop one client creating millions of links (rate limiting)?

Building blocks to know

• Load balancer: spreads requests across servers and stops sending to unhealthy ones.
• Cache: fast, temporary copy of data; you must decide when entries expire or are invalidated.
• Replication: copies of the database for read capacity and failover.
• Sharding (partitioning): splitting data across several databases by a key, when one machine cannot hold or serve it all. It adds real complexity, so it comes late.
• Queue: lets one part of the system hand work to another without waiting (sending emails, generating thumbnails).
• CDN: serves static files from servers close to the user.

Common mistakes

• Drawing boxes before writing requirements and numbers.
• Designing for a billion users when the estimate says ten thousand. Start simple; show how it would grow.
• Forgetting failure: what happens when a server, the cache or the database goes down?
• Keeping session state on one app server, which breaks load balancing.

Recap

• Requirements, estimates, API, data model, diagram, bottlenecks: in that order.
• Back-of-the-envelope numbers decide whether you need a cache, replicas or sharding.
• Stateless servers behind a load balancer scale horizontally; caches protect the database.

Try it

In the Do tab, design a small service on paper using the same six steps.