When the data grows
Five common shapes of code, timed as the input grows from 10 items to 10 million. Each line is how long the work takes: the flat ones stay instant forever, the steep ones stop finishing.
Assumes one operation = 10 ns
Code that works at n = 10 can fall over at n = 10,000,000.
Live pictures of what happens in between.
latency now 20ms
p99, last 10 s 20ms
healthyBuilt-in model
op 10 ns n up to 10M
- O(1)
- O(log n)
- O(n)
- O(n log n)
- O(n²)
| Class | What it looks like in code | Operations | Wall time | Status |
|---|---|---|---|---|
| O(1) | a hash lookup, an array index | 1 | 10 ns | instant |
| O(log n) | binary search, a B-tree index probe | 3 | 33.2 ns | instant |
| O(n) | one pass, a full table scan | 10 | 100 ns | instant |
| O(n log n) | merge sort, an ORDER BY with an index | 33 | 332 ns | instant |
| O(n²) | a loop inside a loop, includes() in a map | 100 | 1 µs | instant |
Sorting a list, five ways
The same 48 numbers sorted by five methods, running for real side by side. Every lane gets the same number of steps per second, so the method that needs fewer steps finishes first, and the slow ones are still grinding long after the fast ones are done.
Real steps, counted as they happen: comparisons, or digit moves for radix sort, which never compares two numbers
Built-in model
Bubble sort O(n²) 0 comparisons sorting
Insertion sort O(n²) 0 comparisons sorting
Quick sort O(n log n) 0 comparisons sorting
Merge sort O(n log n) 0 comparisons sorting
Radix sort O(n) 0 digit moves sorting
sorting 48 numbers table 1,000,000 step 10 ns
| Method | Steps | Time |
|---|---|---|
| Bubble sort | 5 × 10¹¹ comparisons | 1.39 h |
| Insertion sort | 2.5 × 10¹¹ comparisons | 41.7 min |
| Quick sort | 2.77 × 10⁷ comparisons | 277 ms |
| Merge sort | 1.99 × 10⁷ comparisons | 199 ms |
| Radix sort | 1.4 × 10⁷ digit moves | 140 ms |
Loading a page from the database
One page with 3 lists on it, where every row needs a related record (say, each order's customer). Lazy loading fetches that record separately for every row: the N+1 query an ORM makes easy to write. The other three fetch the same data in a handful of trips, whatever the row count.
Assumes each database query costs a 0.8 ms round trip plus 0.15 ms to run, one after another
Built-in model
round trip 0.8 ms query 0.15 ms lists 3
- faint tick: one trip to fetch a list
- solid tick: one trip to fetch related rows
| Approach | Round trips so far | Page load time |
|---|---|---|
| Lazy loading | 97.9 ms | |
| Batched IN | 5.7 ms | |
| Dataloader | 3.8 ms | |
| One join per list | 2.85 ms |
Batched IN and the dataloader both make one extra trip for related rows; the difference is when. Batched IN makes it after each list (3 extra trips); the dataloader waits until every list has loaded and makes it once for the whole page. Lazy loading waits 34× longer than the joins for the same page.
When the server gets busy
How long a request takes as traffic rises, for three database connection pool sizes. Each line sits near 20 ms while there is headroom, then goes vertical, but how much headroom depends on the size of the pool: the pool of 2 is already at 56 ms when 80% busy, while the pool of 32 is still at 29 ms at 95%. The dashed line is a 250 ms target.
Assumes 20 ms of work per request and randomly timed arrivals
Built-in model
work 20 ms pools 2 8 32 SLA 250 ms
- one line per connection pool; the number on each wall is its size
| Connection pool | Max throughput | Busy | Response time | Status |
|---|---|---|---|---|
| 2 connections | 100/s | 10% | 20.2 ms | within SLA |
| 8 connections | 400/s | 3% | 20 ms | within SLA |
| 32 connections | 1,600/s | 1% | 20 ms | within SLA |
Everything holdsup.dev will show
The map of the site. Topics marked built are live on their section's page and the card takes you straight there; the rest are on the workbench. The tag on each card is the shape its curve makes, in the same colours the charts use.
Algorithms & data structures
The classroom material, shown at sizes the classroom never uses.
- every shapebuilt
When the data grows
Five shapes of code timed from 10 items to 10 million, and the one that stops finishing.
- n log nbuilt
Sorting algorithms
Bubble, insertion, quick, merge and radix on the same array, every step counted as it happens. Radix never compares two numbers at all.
- logbuilt
Searching
Linear scan vs binary search vs hash lookup. At a million rows one is 1,000,000 steps, one is 20, one is 1.
- quadraticbuilt
The wrong container
array.includes() inside a loop is quadratic and looks completely innocent. A Set makes it linear.
- quadraticbuilt
String building in a loop
Concatenation copies the whole string every iteration. A builder does not: about 16,000x apart in bytes copied at 100k pieces.
- exponentialbuilt
Recursion without memoisation
Naive Fibonacci against a memoised one: the call tree drawn as it explodes, then collapsing to a line.
- cliffbuilt
Space, not time
Ten million rows in a list vs streaming them. A RAM bar filling toward the ceiling, then the OOM kill.
- linearbuilt
Big-O lies about constants
A linked list and an array are both O(n). Once the list is scattered across memory, walking it can be 100x slower. Cache locality drawn as memory jumps.
- quadraticbuilt
Nested loops over one set
Deduplicating a list by comparing every item with every later one. The triangle of comparisons drawn as an actual triangle, next to the one-pass Set that finishes first.
Databases
Where most real production incidents actually live.
- linearbuilt
N+1 queries
Lazy loading turns one page into N+1 round trips. The join, the batched IN and the dataloader side by side.
- linearbuilt
The missing index
Sequential scan vs index scan, with rows examined as the visual. Add the index and 4,000,000 rows read becomes 8: three index pages, then straight to the matches.
- linearbuilt
OFFSET pagination
Page 1 is instant; page 10,000 reads 200,000 rows to throw away 199,980. Keyset pagination is flat forever.
- linearbuilt
COUNT(*) for a page counter
The "showing 1-20 of 4,312,908" badge costs more than the query it decorates.
- linearbuilt
SELECT * and payload width
Bytes over the wire against columns actually rendered, including the TOAST blob nobody remembered.
- quadraticbuilt
Cartesian explosion
One missing join condition turns 10k x 10k into 100,000,000 rows.
- cliffbuilt
Lock contention on a hot row
A thousand writers, one counter row. Throughput hits a ceiling, then every extra writer only adds wait.
- peakbuilt
Index write amplification
Every index makes reads faster and writes slower. Two curves crossing — the shape of most tuning decisions.
- cliffbuilt
Connection pool exhaustion
500 app threads, 100 database connections. Where the queue forms and why the error is a timeout.
APIs & the network
Latency is a tax on conversations, not on bytes.
- cliffbuilt
The latency cliff
Flat at 20 ms while there is headroom, then vertical. How much headroom you get depends on the size of the box — a small one is already struggling at 80% busy, a big one holds to 95%.
- linearbuilt
Chatty vs batched
Twelve calls one after another vs six at a time vs one batched call, with the distance to the server as the slider (80 ms per round trip is about 5,300 km).
- cliffbuilt
p50 is a lie
A histogram where the average sits inside the SLA, the p99 is three seconds and the p99.9 is eight. Which users those are.
- cliffbuilt
Tail amplification
A page calling twelve services waits for the slowest, so each service's 1-in-100 slow reply lands on 11% of page loads. The maths that surprises people most.
- exponentialbuilt
Retry storms
A blip, then every client retrying, then an outage that outlasts it. Then the same run with jitter and a breaker: calls fail fast for a moment and it recovers within seconds.
- exponentialbuilt
Payload nesting
?expand=orders.items.product with no depth limit. Response size as a tree growing off the screen.
- linearbuilt
Serialisation cost
JSON stringify and parse from 1 KB to 100 MB, timed live in your browser up to 1 MB and extrapolated above, and the size at which gzip's CPU buys back more wire time than it costs.
- stepbuilt
Cold starts
A step function, not a curve. p99 as a function of traffic pattern rather than traffic volume.
Caching, queues & concurrency
Where good intuitions go wrong the most reliably.
- cliffbuilt
The 95% hit rate
95% hits means the 5% owns your p99 and the database still sees 50 rps at peak.
- cliffbuilt
Cache stampede
One popular key expires, every request that misses hits the database, and past about 730 requests a second it never recovers. With and without a lock.
- linearbuilt
Little's Law
L = λW. Why a 5% mismatch between arrival and service becomes an unbounded backlog, and how long the drain takes.
- peakbuilt
Universal scalability
Add workers, throughput rises, flattens, then falls. Coherency cost eating the gains — the most useful curve on the site.
- peakbuilt
Batch size tuning
Latency per event is a U and throughput a peak: batches too small drown in per-write overhead, too big wait to fill and write slowly. The best size is not "as big as possible".
- linearbuilt
Backpressure vs dropping
What a queue does when it cannot keep up: grow, block or shed. Three simulations, one slider.
- peakbuilt
Thread pool sizing
Too few and the cores sit idle, too many and they spend a fifth of their time switching while each request takes 30x longer. The curve has a top.
Architecture, cost & failure
The same curves with dollars and uptime on the y-axis.
- cliffbuilt
Vertical vs horizontal
Two cost curves crossing. The bigger box is cheaper right until it is the most expensive decision you ever made.
- stepbuilt
Replicas, then shards
What each step buys, what it breaks, and the replication lag you inherit in exchange.
- cliffbuilt
Hot partitions
Sharding assumes even keys. One celebrity user and one shard can carry 40% of traffic, however many shards you add.
- linearbuilt
Restore time
The backup is fine. Restoring 4 TB takes nine hours and your RTO says one.
- linearbuilt
Cost per million requests
Two versions of the same endpoint, priced per request: an N+1 with SELECT * costs about 51 times as much every month, and nearly all of it is bytes sent.
- exponentialbuilt
Availability multiplies
Twelve dependencies at 99.9% each is 98.8% — four days of downtime a year.