Recap & real-world examples

Last reviewed: 2026-10 · Volatility: stable

TL;DR

This track covered 28 real-world systems across eight lessons, from single-server scaling patterns to microsecond-latency stock exchanges. The systems span six categories: web-scale services, real-time communication, storage, geospatial, data infrastructure, and financial/transactional. But the same dozen patterns appear again and again. This recap maps every system at a glance. It names the recurring patterns that connect them, and distills the practitioner-level takeaways that cut across all 28 designs.

🎯 For the technical PM

Why it matters — Seeing the patterns that repeat across 28 systems gives you a vocabulary for evaluating any new design. You stop treating each system as a special case and start recognizing the shared building blocks and tradeoffs.

What it changes in your decisions — When your team proposes an architecture, you can identify which of these 28 systems it most resembles. You can ask whether they've addressed that system's known failure modes, and check whether they've chosen the right consistency/availability tradeoff.

Ask your eng team — "Which of these established system designs is ours closest to, and what did we learn from their known failure modes?"

Risk if ignored — You reinvent solutions to problems that were solved a decade ago. You miss failure modes that are well-documented in existing designs, and make avoidable tradeoff mistakes.

---

The systems at a glance

#	System	Lesson	Scale	Core tradeoff	Key mechanism
1	Scaling (single to millions)	Foundations	0 to 10M+ users	Vertical vs. horizontal scaling	Stateless web tier, sharding
2	Back-of-envelope estimation	Foundations	Any	Precision vs. speed of analysis	Power-of-2 reasoning, latency table
3	Four-step framework	Foundations	Any	Breadth vs. depth in design	Scope, sketch, deep-dive, wrap-up
4	Rate limiter	Building blocks	Millions of requests/sec	User experience vs. system protection	Token bucket, sliding window
5	Consistent hashing	Building blocks	N nodes, dynamic membership	Even distribution vs. simplicity	Virtual nodes on a hash ring
6	Key-value store	Building blocks	Billions of keys	Consistency vs. availability (CAP)	Quorum reads/writes, vector clocks
7	Unique ID generator	Building blocks	Millions of IDs/sec	Sortability vs. coordination-freedom	Snowflake: timestamp + machine + sequence
8	URL shortener	Web-scale	100M URLs	Read vs. write optimization	Base-62 encoding, 301/302 redirect
9	Web crawler	Web-scale	Billions of pages	Freshness vs. politeness	Priority queue, URL frontier, robots.txt
10	Notification system	Web-scale	Millions of pushes/min	Delivery reliability vs. latency	APNs/FCM, retry with backoff
11	News feed	Web-scale	100M+ DAU	Fanout-on-write vs. fanout-on-read	Hybrid: push for normal, pull for celebrities
12	Chat system	Real-time	Millions concurrent	Consistency vs. real-time delivery	WebSocket, message queue, read receipts
13	Search autocomplete	Real-time	100K QPS	Freshness vs. latency	Trie in memory, async updates
14	Video streaming	Real-time	Millions concurrent streams	Quality vs. bandwidth/cost	Adaptive bitrate, DAG transcoding, CDN
15	Google Drive (file sync)	Storage	50M users, 10GB each	Sync speed vs. bandwidth	4MB blocks, delta sync, long polling
16	S3-like object storage	Storage	100 PB	Durability vs. storage cost	Erasure coding (8+4), WAL merging
17	Proximity service	Location	200M businesses	Query speed vs. index complexity	Geohash + 8-neighbor lookup
18	Nearby friends	Location	10M concurrent	Real-time accuracy vs. resource cost	WebSocket + Redis pub/sub
19	Google Maps	Location	1B DAU	Tile freshness vs. CDN efficiency	Hierarchical routing, pre-rendered tiles
20	Distributed message queue	Data infra	Millions msgs/sec	Durability vs. throughput	WAL, ISR replication, consumer groups
21	Metrics monitoring	Data infra	10M metrics	Resolution vs. storage cost	Down-sampling, double-delta encoding
22	Ad click aggregation	Data infra	1B clicks/day	Accuracy vs. latency	Kappa architecture, watermarking
23	Distributed email	Data infra	100K emails/sec	Deliverability vs. throughput	SPF/DKIM, IP warm-up, Cassandra by user
24	Hotel reservation	Transactional	~3 TPS	Simplicity vs. concurrency	DB constraints, idempotency key
25	Gaming leaderboard	Transactional	5M DAU	Memory vs. scale	Redis sorted sets, O(log N) ZRANK
26	Payment system	Transactional	Millions txn/day	Safety vs. speed	Double-entry ledger, PSP idempotency
27	Digital wallet	Transactional	1M TPS	Consistency vs. throughput	Event sourcing + CQRS + Raft
28	Stock exchange	Transactional	1B orders/day	Latency vs. distribution	Single-server mmap, custom sequencer

---

Recurring patterns

These patterns appear across many of the 28 systems. Recognizing them lets you transfer understanding from one system to another.

1. Separate metadata from data

The most universal pattern: store the catalog of what exists (metadata) separately from the content itself (data), because they have different consistency, access, and scaling requirements.

System	Metadata	Data
Google Drive	File tree, versions, sharing (MySQL)	Blocks in cloud storage
S3 object storage	Bucket/object catalog (sharded RDBMS)	Blobs on data nodes
Distributed email	Headers, folders, labels (Cassandra)	Body/attachments in blob store
Video streaming	Video info, processing status (RDBMS)	Encoded segments on CDN

2. Append-only logs (WAL)

Write-ahead logs appear as the foundational data structure in systems that prize write throughput and durability:

• Kafka — partitions are append-only logs on disk.
• S3 data nodes — small objects are appended to WAL files.
• Stock exchange — the event log of all orders and trades.
• Digital wallet — event sourcing stores every balance change as an appended event.
• Key-value store — LSM trees use WALs for crash recovery.

The pattern: convert random writes to sequential appends, gain throughput, and accept the cost of background compaction.

3. Idempotency keys

Every system where a retry is possible (which is every system that uses a network) must make writes idempotent:

• Hotel reservation — reservation_id as unique constraint.
• Payment system — payment_order_id as PSP idempotency key.
• Ad click aggregation — (ad_id, window_start) for deduplicated counts.
• Notification system — deduplication of push notifications by event ID.

The pattern: the client generates a unique key before the first attempt. The server uses it to detect and suppress duplicates.

4. Consistent hashing for shard distribution

Consistent hashing distributes data across nodes while minimizing reshuffling when nodes join or leave:

• Key-value store — partition keys across storage nodes.
• Nearby friends — distribute Redis pub/sub channels across a Redis cluster.
• Distributed email — Cassandra ring membership.
• Digital wallet — shard user balances across database instances.
• Message queue — assign partitions to brokers.

5. Event sourcing for auditability

When correctness and history matter more than raw performance, systems store events rather than state:

• Stock exchange — deterministic replay of every order for disaster recovery.
• Digital wallet — complete transaction history for regulatory compliance.
• Ad click aggregation — raw events retained for reprocessing when logic changes.
• Payment system — double-entry ledger entries are effectively events.

The cost: replaying billions of events is slow without periodic snapshots, and the storage footprint grows indefinitely.

6. Pub/sub for fan-out

When one event must reach many consumers, pub/sub decouples the producer from the consumer count:

• Nearby friends — Redis pub/sub channels per user.
• News feed — fanout-on-write pushes posts to follower inboxes.
• Metrics monitoring — Kafka topics fan out to dashboards, alerting, and archival.
• Stock exchange — UDP multicast for market data distribution.
• Notification system — event bus to APNs/FCM/SMS/email channels.

7. Tiered storage (hot/warm/cold)

Not all data is equally accessed. Tiering reduces cost without sacrificing access to recent data:

• Google Drive — SSD for recent files, HDD for older, archival for 90+ day dormant.
• S3 object storage — storage classes (Standard, Infrequent Access, Glacier).
• Metrics monitoring — full resolution for 7 days, down-sampled for months/years.
• Message queue — retention policy moves old segments off primary storage.

8. Geospatial indexing

Converting 2D coordinates to 1D sortable keys enables spatial queries on standard indexes:

• Proximity service — geohash prefix queries.
• Google Maps — tile addressing by zoom/x/y (geohash-like hierarchy).
• Nearby friends — location cache keyed by geohash for bootstrapping.

9. Real-time via persistent connections

When latency matters more than resource efficiency, persistent connections (WebSocket, long polling) replace HTTP polling:

• Chat system — WebSocket for message delivery.
• Nearby friends — WebSocket for location updates.
• Google Drive — long polling for sync notifications.
• Search autocomplete — persistent connection for keystroke-level latency.

10. Exactly-once via idempotent processing

Exactly-once delivery is impossible in a distributed system, but exactly-once processing is achievable by combining at-least-once delivery with idempotent consumers:

• Kafka — idempotent producer + transactional consumer.
• Ad click aggregation — deduplicated writes keyed by (ad_id, window).
• Payment system — PSP idempotency key prevents double charges.
• Hotel reservation — unique constraint on reservation_id.

---

The practitioner's takeaway

Start with the constraints, not the technology

Every system in this track began with the same questions: What's the QPS? What's the storage? What's the latency budget? What's the consistency requirement? The answers — not the latest database trend — drive the architecture. A hotel reservation system at 3 TPS doesn't need Kafka. A stock exchange at 100K orders/sec doesn't need Kubernetes.

Correctness and performance are in tension — pick explicitly

The systems divide cleanly into two camps:

• Correctness-first (payments, wallets, exchanges, reservations): accept higher latency and lower throughput for ACID, idempotency, and double-entry guarantees.
• Throughput-first (metrics, click aggregation, crawlers, news feed): accept eventual consistency and tolerate occasional duplicates or losses.

The worst designs are the ones that don't choose — they promise both and deliver neither.

The metadata store is always the critical path

Across Google Drive, S3, email, video streaming, and the stock exchange, the metadata store is the single most critical component. Data can be replicated and reconstructed. Metadata loss means the system doesn't know what it has. Invest disproportionately in metadata store durability, replication, and backup.

Every retry is a potential duplicate

The moment you add a network between two components, you accept that operations may execute more than once. If you haven't made every write operation idempotent, you haven't designed a production system — you've designed a demo.

Scaling is a sequence, not a choice

The foundations lesson laid out the scaling sequence: separate compute from storage, add load balancing, add replication, add caching, add sharding. Every system in this track follows some prefix of this sequence. The art is knowing where to stop. Most systems don't need the full sequence, and jumping ahead — sharding before you've cached, caching before you've indexed — creates avoidable complexity.

The gap between "works" and "works correctly at scale" is where careers are made

A URL shortener, a chat system, or a payment flow can be built in a weekend hackathon. Making it handle 100M users, survive node failures, reconcile every transaction, and maintain sub-200ms latency — that's the work this track is about. The designs here aren't clever algorithms. They're decades of production experience encoded as patterns and checklists.

---

Test yourself

1. Adding an 11th server to 10 moves 91% of keys with hash mod N. How many move on a ring with 100 virtual nodes per server?
   

   Answer

   About 9.7%, close to the ideal 1/11. Virtual nodes also even out the load: the busiest server held 1.08 times the average, against 2.69 times with one point per server. (Core building blocks)

2. A token bucket has a rate of 5 per second and a capacity of 10. What do 20 simultaneous requests get?
   

   Answer

   10 are allowed and 10 are refused. The bucket holds 10 tokens, and refills are too slow to help within the same instant. (Core building blocks)

3. Why can a fixed-window rate limiter allow twice the limit?
   

   Answer

   Requests just before and just after a window boundary count in different windows. In the example, 200 requests in a burst were all allowed, against 101 for a sliding-window counter. (Core building blocks)

4. With 3 replicas, which of W = 1 R = 1, W = 2 R = 2 and W = 3 R = 1 always read the latest write?
   

   Answer

   W = 2 R = 2 and W = 3 R = 1. Both satisfy R + W > N, so every read set overlaps every write set. W = 1 R = 1 does not. (Core building blocks)

5. What should a Snowflake-style ID generator do if its clock moves back?
   

   Answer

   Refuse to issue IDs, or wait, until time passes the last timestamp. Issuing IDs would risk duplicates. (Core building blocks)

6. A feed has 100 celebrity accounts with 20 million followers each. Why is pure fan-out on write a poor fit?
   

   Answer

   The celebrities alone caused 98% of about 10.2 billion daily feed writes. A hybrid, which pulls celebrity posts at read time, cost about 1.7 billion operations. (Web-scale services)

7. 1,000 requests arrive at the moment a hot cache key expires. How do you keep the database safe?
   

   Answer

   Coalesce requests so one request refills the cache and the rest wait: 1 read instead of 1,000. Add jitter to expiry times so keys do not expire together. (Web-scale services)

8. At 100 million new links a day, how long do 7 Base62 characters last?
   

   Answer

   About 96 years. 62 to the 7th power is about 3.52 trillion codes. (Numbers and formulas)

9. A message log delivers at least once, and a consumer crashes after processing a message but before committing its offset. What protects you?
   

   Answer

   An idempotent consumer. Without it the example charged 80 instead of 60. (Data infrastructure)

10. A user retries a chat send because the ack was lost. How do you avoid a duplicate message?
    

    Answer

    The client sends the same message ID, and the server returns the stored sequence number instead of storing the message again. (Real-time systems)

11. A user edits one byte of a 400 KB file. How much does a block-based sync client upload?
    

    Answer

    One 4 KB block, because the client hashes blocks and skips the 99 the server already has. (Storage and sync)

12. How does a grid index make nearby search fast?
    

    Answer

    It maps each location to a cell and searches only the rider's cell and its neighbours. In the example, 190 of 50,000 drivers were examined and none was missed. (Location and geo services)

13. A payment is retried. What stops the ledger from posting it twice, and what invariant catches a malformed entry?
    

    Answer

    An idempotent transaction ID, and the rule that debits and credits in a transaction sum to zero. (Transactional and financial systems)

14. A 99.99% SLO allows how much downtime in a 30-day month, and what do two 99.9% services in series give?
    

    Answer

    About 4.32 minutes. In series, two 99.9% services give about 99.8%. (Numbers and formulas)

15. For 200 million daily users with 2 posts each, what write rate and yearly storage do you estimate?
    

    Answer

    About 4,630 writes per second on average and about 43.8 TB a year for 300-byte posts. Reads, at 100 times writes, are about 463,000 per second. (Foundations and framework)

16. Round robin and least connections are compared with every third request slow. What happens?
    

    Answer

    Round robin sends all the slow requests to one server, which reached 17 open connections. Least connections peaked at 6. (Traffic, caching, APIs and storage choice)

17. A write-back cache flushes every 100 writes and crashes after write 457. What is lost?
    

    Answer

    The 58 acknowledged writes that were not yet flushed. A write-through cache would lose none, at the cost of slower writes. (Traffic, caching, APIs and storage choice)

18. Two new rows arrive while a user reads a feed. Why does offset paging repeat rows, and cursor paging not?
    

    Answer

    An offset counts positions, so inserting rows at the front shifts every row down. A cursor asks for rows older than the last one seen, so it does not move. (Traffic, caching, APIs and storage choice)

19. A user changes a photo and refreshes half a second later and sees the old one. Name the violated guarantee and a fix.
    

    Answer

    Read-your-writes. Read from the leader for a short window after the user's own write. (Distributed data and consistency)

20. A trip saga reserves a flight and a hotel and then the card charge fails. What happens next?
    

    Answer

    The saga compensates in reverse order: it cancels the hotel, then the flight. (Distributed data and consistency)

21. Why does a transactional outbox beat updating the database and then publishing an event?
    

    Answer

    A crash between the two steps loses the event. With an outbox the event is stored in the same transaction, and a relay publishes it later. In the example, 9 of 10 events were published with a dual write and 10 of 10 with an outbox. (Distributed data and consistency)

22. Why can last-write-wins by wall-clock timestamp drop the newest write?
    

    Answer

    Clocks drift. A server running 2 seconds fast stamps an earlier write later than a later write on a correct clock. Logical clocks order by cause and effect instead. (Distributed data and consistency)

23. A service handles 100 requests per second and is offered 90. A one-second blip makes every call fail, and clients retry three times. Why does it not recover?
    

    Answer

    The retries raise the offered load to 360 per second, so the service stays overloaded. Backoff, jitter and a retry budget kept the peak at 96 per second. (Reliability and operations)

24. What did the Cloudflare 2019 outage and the GitLab 2017 outage have in common with a missing safeguard?
    

    Answer

    Cloudflare pushed a rule to the whole network with no staged rollout and no CPU check in the tests. GitLab had four recovery mechanisms and none worked, because nobody owned testing the restore. (Reliability and operations)

25. Why does a funnel retrieve 500 of 20,000 items before the heavy model runs, and what is the risk?
    

    Answer

    It cut the cost from 2,000,000 units to 70,000, about 29 times less. The risk is that an item dropped at retrieval can never be recovered, so measure the recall of the retrieval stage. (ML system design)

26. A model's click probabilities are all three times too high. Does the ad order change, and does it matter?
    

    Answer

    The order does not change, and it still matters: predicted clicks are three times actual clicks, and every impression is mispriced. Calibration matters as much as ranking when the score sets a price. (ML system design)

27. What do you trade when you probe more buckets in an approximate nearest-neighbour index?
    

    Answer

    Recall against work. In the example, probing 1, 4 and 16 of 64 buckets scanned 2%, 6% and 26% of the vectors and found 24%, 60% and 94% of the true top 10. (ML system design)

28. Why does grouped-query attention let one accelerator serve more long conversations?
    

    Answer

    It shrinks the KV cache per conversation. In the example, 16 conversations fit with 32 key-value heads and 68 with 8. (GenAI and agent platforms)

29. Why is continuous batching faster than static batching?
    

    Answer

    A finished request leaves at once and a waiting one takes its slot, so the accelerator does not idle waiting for the longest request. In the example, 1,887 steps against 2,890. (GenAI and agent platforms)

30. An agent crashes after its third of five tool calls. How do you avoid repeating paid calls?
    

    Answer

    Checkpoint progress after each committed step and resume from it. In the example, five calls were made in total with no repeats. Also add budgets, per-tool permissions and a kill switch. (GenAI and agent platforms)

31. 50,000 likes arrive in one second on a viral post. How do you avoid a hot row?
    

    Answer

    Shard the counter and sum the shards on read. In the example, the busiest row fell from 50,000 writes to 3,195 with 16 shards. Deduplicate (user, post) pairs so double taps count once. (Social and media systems)

32. How can you count unique viewers of a post in a few kilobytes?
    

    Answer

    With a HyperLogLog. Redis's version uses up to 12 KB and has a standard error of 0.81%. In the example, 100,000 distinct items were counted with an error of 0.42%. (Social and media systems)

For more practice, use the question bank.

---

The full track

1. Foundations & framework — scaling, estimation, the four-step design process, and a practice plan
2. Numbers, formulas and papers — latency, nines, estimation shortcuts and classic papers
3. Traffic, caching, APIs and storage choice — load balancers, caches, CDNs, stores and APIs
4. Core building blocks — rate limiting, consistent hashing, key-value stores, unique ID generation
5. Distributed data and consistency — replication, consistency, consensus, sagas and clocks
6. Web-scale services — URL shortener, web crawler, notification system, news feed
7. Real-time systems — chat, search autocomplete, video streaming
8. Storage & sync — Google Drive file sync, S3-like object storage
9. Location & geo services — proximity service, nearby friends, Google Maps
10. Data infrastructure — message queues, metrics monitoring, ad click aggregation, email
11. Transactional & financial systems — hotel reservation, leaderboard, payments, digital wallet, stock exchange
12. ML system design — recommendation funnels, ads click prediction, search and vector indexes
13. GenAI and agent platforms — LLM serving, continuous batching, agent runtimes and safety
14. Social and media systems — likes, graph search, matching, photo tiers, video and live streaming
15. Reliability and operations — SLOs, resilience, safe change, recovery and incident case studies
16. Question bank — 50 design questions mapped to the lessons

Sources

• Alex Xu, System Design Interview, volumes 1 and 2, and Martin Kleppmann, Designing Data-Intensive Applications, O'Reilly, 2017.
• The sources for each system are at the end of its lesson. The numbers in the questions come from system-design/code/. They are invented or estimated. These sources were not re-opened during this revision, and the module's fact-check is pending.