How this handbook teaches

The goal is not to memorize 16 architectures. The goal is to look at a new prompt, name what makes it hard, and assemble a design from building blocks you understand well enough to defend. So the handbook asks you to think before it explains.

• Try before you read. Every lesson opens with a short question. Answer it on paper first; getting it wrong is what makes the explanation stick.
• Analogy, then mechanism, then trade-off. Each idea starts with what it solves and a plain analogy, then how it works, then what it costs. Depth sits in folded "Go deeper" sections.
• Step through every flow. Diagrams with a step bar reveal one arrow at a time. Predict the next arrow before pressing Next. You should be able to redraw each one from memory.
• Production notes. Green "In production" boxes show how the idea appears in real systems, including payments and data pipelines, so the handbook serves your day job, not only interviews.
• Interview signals. Purple boxes say what interviewers listen for at SDE-2 (L4) and what an L5 answer adds.
• One method everywhere. The decode method turns any prompt into pressures and building blocks in seven steps.
• Live tools. Calculators for estimation and availability, a quorum explorer, a consistent hashing playground, and a token bucket simulator let you test intuitions with numbers.

How to study one module

1. Clear the revision queue above at the start of each session: redesign those systems from a blank page.
2. Do the warm-up quiz without reading anything. Note which questions you missed.
3. For each lesson: answer the "Try it first" question, read the short part, step through the figure, then say out loud what problem the idea solves and when it fails.
4. Open "Go deeper" only when a case study or your own question needs it.
5. Do the self-check quiz at the end, then tick lessons as studied.
6. Tick the mastery checklist only when each item is true with your notes closed.

How to practice a case study

1. Set a 45-minute timer and design it yourself on paper or a whiteboard using the interview framework: requirements, estimates, API, data model, diagram, deep dives.
2. Then open the case study and compare section by section. Answer each guided deep dive before revealing it.
3. Set an honest status at the top of the page. The handbook schedules a review.
4. Redesign it from memory when it comes due, faster each time. Then try the related practice prompts in the tracker.

Design statuses and the revision queue

When a design comes up for review, redesign it without looking at notes, then set the new status or press "Reviewed, same status" to restart its timer.

Where your progress is saved

Lesson ticks, mastery checks, readiness checks, and design statuses save automatically to your account, privately: nobody else who opens this page sees them, and they follow you to other devices. The indicator in the top bar shows where the latest change went. If the page cannot reach your account (for example in a private window), it saves in this browser instead, and the tracker has a backup and restore box.

Scope

This handbook covers what general SDE-2 system design rounds test, in the depth they test it, plus the production knowledge a backend engineer uses daily. It deliberately leaves out: machine learning system design (feature stores and model serving appear only as boxes), cloud certification trivia, and research-level internals (Paxos proofs, CRDT algebra). Low-level design (classes and patterns) belongs to a separate LLD track.

The map

Priority overview

Total: about 45 hours of study, plus about 20 hours for the 16 case studies (45 minutes to design each, then about 30 minutes to compare and note what you missed).

How each module is organized

• Warm-up quiz to expose what you do not know yet.
• Lessons, each opening with a "Try it first" question, then what the idea solves, an analogy, the mechanism, and the trade-offs.
• Figures that step through flows one arrow at a time.
• Go deeper folds for details you need only sometimes.
• In production notes and Interview signal notes.
• Common mistakes, a self-check quiz, and a mastery checklist.

Case studies follow one format: try it first, requirements, estimates, API, data model, architecture figure, guided deep dives, failures, what a strong answer shows at L4 and L5, and follow-up questions.

Almost every design is assembled from about a dozen blocks. Learn what each one is for, and you can read any architecture diagram.

The rule this figure teaches: keep the synchronous path short. Everything the user waits for adds latency and is a place to fail. Move anything slow or non-essential (emails, thumbnails, analytics) to a queue.

The NFR checklist to run through in every design:

The trade-offs you will name in almost every design:

• Consistency vs availability and latency. Strong consistency needs coordination, which costs round trips and can block during failures (Module 11).
• Latency vs throughput. Batching raises throughput but makes each item wait (Kafka producer batching, DB group commit).
• Read speed vs write speed. Indexes, caches, and denormalized copies make reads fast and writes slower or more complex.
• Simplicity vs scale. A single PostgreSQL primary handles far more than most people assume. Shard only when numbers force you to.

DNS is also a coarse load balancer:

• Round-robin DNS: return several IPs; clients spread across them. No health awareness.
• GeoDNS / latency-based routing: return the IP of the region closest to the user (Route 53 latency routing, Cloud DNS geolocation).
• Failover routing: health-check a region and stop returning its IP when it fails.

TLS (the S in HTTPS) gives three things: encryption (nobody can read the traffic), integrity (nobody can modify it), and authentication (the certificate proves the server is really shop.com). TLS 1.3 needs one round trip; TLS 1.2 needed two.

Scaling persistent connections

Each WebSocket is a long-lived connection pinned to one server. Two problems follow: the load balancer must keep each client on its server for the life of the connection, and a message for Bob must reach the specific server holding Bob's connection.

• gRPC = Protocol Buffers (schema and binary encoding) + HTTP/2 (transport). Four call types: unary, server streaming, client streaming, bidirectional streaming.
• Strengths: performance, strict contracts, code generation in many languages, deadlines that propagate across calls.
• Costs: not human-readable, browsers need a gRPC-Web proxy, and long-lived HTTP/2 connections need L7 or client-side load balancing.
• Typical split: REST/JSON for public and browser-facing APIs, gRPC between internal services. API styles are compared in Module 3.

Rules that keep an API clean:

• Nouns in paths, verbs in methods: POST /orders, not POST /createOrder. For actions that are not CRUD, use a sub-resource: POST /orders/9/cancel.
• The user comes from the auth token, not from the body: GET /feed, not GET /feed?user_id=7. Otherwise anyone can read anyone's feed.
• Every list is paginated and has a maximum page size.
• Return what the client needs next: IDs, a cursor, a status URL.
• Use IDs that do not leak information: random or time-ordered IDs, not sequential ones that reveal your order count (Module 17).

The cursor is usually an opaque, encoded value of the sort key, for example base64 of (created_at, id). Include the ID as a tie-breaker so rows with equal timestamps are not skipped.

• Version in the path (/v1/orders) is the most common and easiest to route; header-based versioning keeps URLs stable but is less visible.
• Prefer additive change inside a version; create v2 only for breaking changes, and run both until clients migrate.
• One error format everywhere: an HTTP status plus a body like {"error": {"code": "card_declined", "message": "...", "request_id": "..."}}. The machine-readable code lets clients branch; the request ID lets support find logs.
• Tell clients about limits: 429 with Retry-After, and headers for the current quota.
• Optimistic concurrency: return an ETag (a version); clients send If-Match on updates; the server returns 412 if someone else changed it first.

Webhooks are the server calling the client. To make them reliable:

• Sign every payload: an HMAC-SHA256 of the body plus a timestamp, using a shared secret. The receiver recomputes it and rejects stale timestamps to stop replays.
• Retry with backoff on non-2xx responses, for hours or days, and give the receiver a way to fetch missed events.
• Receivers must be idempotent: the same event can arrive twice. Deduplicate by event ID.
• Receivers should answer fast: store the event, return 200, process asynchronously.

Rough capacity per node

These are planning numbers for simple operations on modern hardware. Real numbers depend on payload size, query shape, and tuning, so say "roughly" and move on.

The formula: QPS = daily active users x actions per user per day / 86,400. Then:

• Split reads and writes. They usually scale differently and hit different components. A 100:1 read-to-write ratio points at caching and read replicas; write-heavy points at partitioning, queues, and append-friendly storage.
• Peak vs average. Use 2-3x for normal daily peaks; 10x or more for flash sales, ticket launches, or live events.
• Fan-out multiplies. One post to 1,000 followers is 1 write request but 1,000 feed updates if you precompute feeds (Case study 4).

Storage = new items per day x size x retention days x replication. Then say what it implies:

• Single-digit TB and a few thousand writes/s: one well-provisioned relational primary with replicas is usually fine.
• Tens of TB, or writes beyond what one primary handles: partition the data (Module 10).
• Large binary objects (photos, video): never in the database. Object storage holds the bytes; the database holds metadata and the object key (Module 7).
• Indexes add space: often 20-100% on top of the raw data.

• Bandwidth = QPS x response size. Gbps = GB/s x 8.
• Cache size: many systems follow an 80/20 shape, where about 20% of items get about 80% of reads. Caching 20% of a day's read volume is a common first estimate and usually an overestimate, since repeated reads hit the same items.
• Servers = peak QPS / QPS per server, then add headroom (run at 50-70% of capacity) and redundancy (survive losing one zone).

• Little's Law: items in a system = arrival rate x time each item spends in it (L = lambda x W). It holds for any stable system, so it sizes thread pools, connection pools, and consumer counts. Consumers needed = message rate x processing time per message (5,000 msg/s x 20 ms = 100 concurrent consumers).
• Utilization drives queueing: as a server nears 100% busy, waiting time explodes. In a simple single-server queue, average wait is about service time x rho / (1 - rho), where rho is utilization:

That curve is why capacity plans target 50-70% utilization at peak, and why latency spikes long before CPU reaches 100%.

Six drills

Do each drill in about 5 minutes on paper. Answer each step before revealing it. The goal is speed at spotting what matters, not a full design.

The method

Use this sequence on every prompt. It is the same order as the interview framework, compressed to what drives decisions.

Signals and the blocks they point to

Quick reflex quiz

Sticky sessions (the LB keeps sending a user to the same server, by cookie or IP hash) is a shortcut that keeps state on servers. It works, but load becomes uneven, and when that server dies its users lose their sessions. Prefer external state; use stickiness only as an optimization (for example to keep a warm local cache), never for correctness.

Health checks keep traffic away from broken servers:

• Active: the LB calls /health every few seconds and removes a server after N failures, adding it back after M successes.
• Passive (outlier detection): the LB watches real traffic and ejects servers that return too many errors or time out.
• Readiness vs liveness (Kubernetes): failing readiness takes a pod out of the load balancer (it is warming up or overloaded); failing liveness restarts it (it is stuck). A health endpoint that checks the database can take every pod out at once when the database blips, so keep readiness checks about the instance itself.
• Connection draining: on deploy or scale-in, stop sending new requests to an instance and let in-flight ones finish before killing it.

• Redundancy at every tier: at least N+1 instances, spread across availability zones (separate power and network within a region).
• Load balancers themselves: cloud LBs are already distributed across zones. Self-run ones use an active-passive pair sharing a virtual IP (keepalived/VRRP): if the active one stops sending heartbeats, the passive one takes the IP.
• Across regions: GeoDNS or anycast sends users to the nearest healthy region (global server load balancing). Region failover also needs the data to be there (Module 15).
• Availability multiplies: two components in series, each 99.9%, give about 99.8%. Two redundant copies in parallel, each 99.9%, give about 99.9999% if their failures are independent. Module 15 has a calculator.

An API gateway is an L7 reverse proxy with API features: authentication and token validation, rate limiting and quotas, routing to services, request and response transformation, TLS termination, request IDs, and sometimes response caching or aggregation. Examples: Kong, Apigee, AWS API Gateway, Envoy-based gateways.

• A backend for frontend (BFF) is a gateway per client type (web, iOS, partner API) that shapes responses for that client.
• Keep the gateway thin. Business logic in the gateway turns it into a bottleneck every team must change.
• The gateway is on every request, so it must be redundant and fast, and it must fail predictably (Module 15).

• Scale on a metric that tracks load: CPU, requests per instance, queue length (best for workers), or p99 latency.
• Target tracking: "keep average CPU near 60%" adds and removes instances to hold the target.
• Cooldowns and limits: avoid flapping (adding and removing repeatedly); set a maximum so a bug or attack cannot scale you into a huge bill.
• Scale-in carefully: drain connections and finish in-flight jobs before terminating.
• Scheduled or predictive scaling for daily peaks and known events.
• Remember the dependencies: doubling app servers doubles the connections to the database. Scale the bottleneck, not just the easy tier.

• Hit ratio = hits / (hits + misses). Database load = miss ratio x read QPS.
• Cache data that is read often, changes rarely, and is expensive to compute or fetch.
• Do not cache data that must be exactly current for correctness (an account balance used to approve a payment) unless the cache is part of a consistent design.

Ways to bound or remove staleness, from simplest:

1. Always set a TTL. It is the safety net that limits any bug or race to N seconds.
2. Delete, do not update, on write. Two concurrent writers updating the cache can apply in the wrong order; a delete cannot.
3. Delayed double delete: delete, then delete again a short time later, to catch a racing reader. Cheap, not a guarantee.
4. Versioned values: store a version with the value and only overwrite with a newer version (a conditional set or Lua script).
5. Leases: on a miss the cache hands the reader a token; a delete invalidates outstanding tokens, so the late set is rejected. Facebook's memcache uses this.
6. Invalidate from the database's change log (CDC, Module 13): a consumer reads committed changes and deletes affected keys, so no write path can forget.

In Redis, maxmemory caps memory and maxmemory-policy picks the policy: allkeys-lru or allkeys-lfu for a pure cache, volatile-* variants to evict only keys that have a TTL. The default is noeviction, which returns errors on writes when memory is full, the right choice when Redis holds data you cannot lose. Redis approximates LRU and LFU by sampling keys, which is cheap and close enough.

How Redis behaves, in the points interviews probe:

• Commands execute on one thread, so each command is atomic. Multi-step logic made atomic with MULTI/EXEC or a Lua script (used for rate limiters, Case study 2). Newer versions use extra threads for network I/O, but command execution stays single-threaded.
• Persistence is optional: RDB snapshots (periodic, may lose minutes) and AOF (append-only log, typically fsync every second, so up to about a second of loss).
• Replication is asynchronous: a failover can lose recently acknowledged writes. Redis Sentinel automates failover for a primary and its replicas.
• Redis Cluster shards keys over 16,384 hash slots (CRC16(key) mod 16384). Multi-key commands work only within one slot; a hash tag like {user:7}:cart forces related keys into the same slot.
• Valkey is the open-source, Redis-compatible fork that many managed cache services now offer; everything here applies to it.
• Memcached is the simpler alternative: multithreaded, strings only, no persistence or replication, sharded by the client. Fine for pure caching; Redis wins when you need data structures.

What to know about object storage (S3 as the example):

• Flat namespace: a bucket holds objects under keys like photos/2026/10/abc.jpg; the slashes are just characters, not real directories.
• Objects are replaced whole: no in-place edits. To change a file, write a new version. S3 also supports conditional writes (If-None-Match: * to create only if absent, If-Match for compare-and-swap), useful for simple locks and manifests.
• Very durable: S3 is designed for 99.999999999% (11 nines) durability by storing data redundantly across multiple facilities.
• Strong read-after-write consistency for S3 since 2020: after a successful PUT, reads see the new object.
• Storage classes and lifecycle rules: move objects to cheaper infrequent-access or archive tiers after N days, and delete them after the retention period.
• Versioning and object lock protect against accidental deletes and support write-once retention for compliance.

• A presigned URL is a normal storage URL plus a signature and expiry, created with the server's credentials. Whoever holds it can do exactly one thing (PUT this key, or GET this key) until it expires.
• Large files go in parts (S3 multipart upload, GCS resumable upload): each part is retried on its own, and parts can upload in parallel.
• Validate after upload, not before: check type, size, and malware in the worker, since the client controls the bytes.
• Downloads of private files use presigned GET URLs or CDN signed URLs, so authorization stays in your API while bytes flow from storage or the CDN.

Controlling what the CDN caches:

• Cache-Control: public, max-age=31536000, immutable for versioned static files (hash in the file name). They never need purging.
• s-maxage sets the TTL for shared caches (the CDN) separately from browsers.
• no-cache means "store it, but revalidate before each use" (with ETag / If-None-Match, a 304 is cheap). no-store means "never store it", for sensitive responses.
• Cache key: by default the URL. Adding headers or cookies to the key (for language or device) splits the cache; never cache per-user responses under a shared key.
• Purge / invalidation removes content early, but takes time to reach all edges. Versioned URLs avoid needing it.
• Stale-while-revalidate: serve the cached copy while fetching a fresh one in the background.

• Images: store the original; serve resized, compressed variants (WebP or AVIF where supported) from the CDN.
• Video: transcode each upload into several resolutions and bitrates and cut each into short segments (2-10 s). A manifest lists the variants; the player switches quality per segment based on bandwidth. This is adaptive bitrate streaming (HLS or MPEG-DASH), and every segment is a cacheable CDN object. Case study 9 designs the full pipeline.

• Storage tiers trade retrieval speed and fees for a lower monthly price. Archive tiers can take minutes to hours to restore.
• Egress is often the biggest bill. Serving through a CDN reduces origin egress; keep compute in the same region as the bucket.
• Encrypt at rest with keys from a key management service (server-side encryption with KMS); restrict buckets to private and grant access through presigned URLs and IAM roles, never public listing.
• Checksums (MD5 or CRC) on upload detect corruption; store them in metadata for later verification.

• Never hold a database lock across a network call to another service; a slow call then blocks everyone.
• Deadlocks happen when two transactions lock rows in opposite orders. Lock in a consistent order (for example by ID) and keep transactions short. Databases detect deadlocks and abort one transaction.
• SKIP LOCKED turns a table into a simple work queue; NOWAIT fails immediately instead of waiting.

• B+tree index: sorted; supports equality, ranges, ORDER BY, and prefix matches. The default in relational databases.
• Composite index (a, b, c): usable for a, a, b, and a, b, c. Put equality columns first, then the range or sort column.
• Covering index: contains every column the query needs, so the table itself is not read (INCLUDE columns in PostgreSQL).
• Partial index: indexes only rows matching a condition (WHERE status = 'pending'), small and fast for hot subsets.
• Hash index: equality only. Inverted index: word to documents, for full-text search (Module 17). Geospatial index: R-tree, geohash (Module 17).
• Cost: every index slows writes and uses space. Index for your actual queries; check them with EXPLAIN.

Questions that decide the choice:

1. What are the access patterns? Lookups by key, range scans by time, ad hoc queries, joins, full-text, graph traversals.
2. Do you need multi-row transactions? Money, inventory, bookings: yes. Likes, views, logs: rarely.
3. What is the read/write ratio and volume? Write-heavy at huge scale favors LSM-based wide-column stores.
4. How big will it get? Single-digit TB and a few thousand writes/s: one relational primary plus replicas. Tens of TB or more writes: partitioning or a natively distributed store.
5. What consistency does each feature need? (Module 11)
6. What does the team already run well? Operational experience is a real requirement.

• Normalize for correctness (one place to update) when writes and consistency matter. Denormalize for read speed when reads dominate and slight staleness is acceptable.
• Model by access pattern for NoSQL: list the queries first, then design one table (or key layout) per query.
• Keep transactions inside one partition where possible: put data that changes together under the same partition key (Module 10).
• Separate OLTP and OLAP: the transactional database serves the product; a warehouse fed by change data capture or batch export serves analytics.
• Soft deletes and audit trails: append-only history tables (who changed what, when) are often required in regulated domains.

• How changes ship: the leader sends its write-ahead log or a logical change stream (PostgreSQL streaming replication, MySQL binlog). Followers apply changes in the same order.
• Read scaling: add followers for read-heavy workloads. Each extra follower adds load on the leader to ship the log, and all of them lag a little.
• Used by: PostgreSQL, MySQL, SQL Server, MongoDB replica sets, Redis, Kafka partitions (each partition has a leader).

Consensus-based systems (Module 11) generalize this: a write commits once a majority stores it, which tolerates a minority of slow or failed nodes without losing acknowledged writes.

• Lost writes: with async replication, acknowledged writes on the old leader that never reached the new one are gone. Their IDs may even be reused, which corrupts other systems that saw them.
• Split brain: prevent it by requiring a majority to elect a leader (a 3-node or 5-node cluster) and by fencing: each leader term has a higher epoch number, and storage or downstream systems reject requests carrying an older epoch.
• Use battle-tested tooling: Patroni (PostgreSQL, using etcd for consensus), managed databases with automatic failover (RDS Multi-AZ, Cloud SQL HA), Redis Sentinel. Do not hand-roll failover.
• RPO and RTO (Module 15) describe the result: how much data you can lose (RPO) and how long recovery takes (RTO).

• Use when: multiple regions each need low-latency writes, clients work offline (each device is a "leader" that syncs later), or collaborative editing.
• Conflict resolution:
• Last write wins (LWW): keep the value with the highest timestamp. Simple, but loses data, and clock skew can pick the "wrong" winner.
• Application merge: union of shopping cart items, max of counters, flag for manual resolution.
• CRDTs (conflict-free replicated data types): counters, sets, and text types designed so concurrent updates always merge to the same result (Case study 16).
• Avoid conflicts: give each record a home region and send all its writes there.
• Cost: much harder to reason about; avoid unless the requirement truly needs it.

Dynamo-style stores (Cassandra, Riak, the original Dynamo; ScyllaDB) let the client write to and read from several replicas directly:

• N replicas per key, W acknowledgments required for a write, R replies required for a read. R + W > N means reads and writes overlap. Common choice: N = 3, W = 2, R = 2 (Cassandra's QUORUM).
• Tunable: W = 1, R = 1 for speed with possible staleness; W = N for reads that only need one replica but writes that fail when any node is down.
• Read repair: when a read sees a stale replica, it writes the newer value back to it.
• Anti-entropy: a background process compares replicas using Merkle trees (hash trees over key ranges) and copies only the ranges that differ.
• Hinted handoff: if a target replica is down, another node stores the write with a "hint" and delivers it when the replica returns. With a sloppy quorum, those stand-in nodes count toward W, which keeps writes available but weakens the overlap guarantee.
• Conflicts still happen with concurrent writes; stores use LWW timestamps (Cassandra) or version vectors and siblings (Riak).

A good partition key:

1. Has many distinct values (high cardinality), so data can spread across many nodes.
2. Spreads load evenly, both data size and request rate. Watch for skew: a few very hot keys.
3. Matches the main access pattern, so common queries hit one partition. The key you always know at query time (user ID, conversation ID) is usually the right one.
4. Keeps things that must change together in one partition, so transactions stay local (an account and its ledger entries).

Compound keys get both: Cassandra hashes the partition key to pick the node, then sorts rows inside the partition by clustering columns. PRIMARY KEY ((user_id), created_at) spreads users evenly and keeps each user's rows in time order (Module 8).

How it works:

1. Hash both nodes and keys onto the same circle of hash values (for example 0 to 2^32 - 1).
2. A key belongs to the first node clockwise from its position.
3. Adding a node takes keys only from its clockwise neighbor; removing a node gives its keys to its neighbor. About K/N keys move (K keys, N nodes).
4. Virtual nodes: each physical node is placed at many points (for example 100-256). This evens out the arcs, spreads a failed node's load across all others instead of one neighbor, and lets bigger machines take more points.
5. Replication on the ring: store each key on the next N distinct physical nodes clockwise (Dynamo, Cassandra).

• Hot keys: celebrities, viral items, a dominant tenant. Fixes: key salting (add a suffix to spread writes, merge on read), caching hot reads, dedicated partitions for known giants.
• Hot ranges: sequential IDs or timestamps in range partitioning. Fixes: hash prefix, random or time-ordered-but-spread IDs, time buckets combined with another key.
• Skewed sizes: one partition grows far larger (a huge group chat). Fixes: split by (key, bucket) such as month, cap partition size.

How a request finds the right node:

1. Any node forwards: the client contacts any node, which forwards to the owner (Cassandra coordinator).
2. Routing tier: a proxy that knows the map (Vitess VTGate for MySQL, MongoDB mongos, a Redis proxy).
3. Partition-aware clients: the client library caches the map and goes straight to the owner (Redis Cluster clients follow MOVED redirects, Kafka clients fetch metadata).

The map itself lives in a coordination service (ZooKeeper, etcd) or is spread by gossip, and moving data happens in the background, throttled so it does not overload the cluster.

Cross-partition joins and transactions are expensive: avoid them through key choice and denormalization, run them in batch or analytics systems, or use a store that supports distributed transactions (Spanner, CockroachDB) and accept the latency. Patterns such as sagas (Module 13) coordinate multi-partition business operations.

• C (in CAP) = linearizability. A = every request to a non-failed node gets a non-error response. P = the system keeps working despite lost messages between nodes.
• Since partitions happen, the real choice is CP or AP during a partition:
• CP: refuse or time out some requests to avoid stale or conflicting data (etcd, ZooKeeper, HBase, Spanner, a single-leader SQL database whose minority side cannot accept writes).
• AP: keep answering and reconcile later (Cassandra and DynamoDB with eventual reads, DNS, multi-leader setups, most caches).
• Many systems are tunable: Cassandra with QUORUM reads and writes behaves much more like CP than with ONE.

PACELC: if there is a Partition, choose A or C; Else, choose Latency or Consistency.