The Most Expensive Sentence in System Design

One decision flow for the five caching strategies, the eviction policy Redis actually ships with, and the three ways a cache takes down the database it was supposed to protect.

"We'll just add a cache" is the most expensive sentence in system design. Utter that sentence, and you instantly inherit three headaches: where writes go, what gets thrown out when memory fills up, and the big one, how you find out your data is a lie.

I used to think the invalidation joke was tired. Then a warm-up script with one shared TTL took down our main Postgres database. The deploy itself was quiet. An hour later every key expired inside the same second.

Strategies and eviction come first because they're the quick decisions. The back half goes to invalidation and the failure modes, since that's what pages you at night. CDNs and cache coherence protocols are out of scope on purpose. Most interview follow-ups I've sat through live there too, so that's where the words go.

The Five Strategies Are Two Decisions Wearing a Trenchcoat

All five strategies in one flow. The top two lanes differ on who fetches data on a cache miss; the bottom three differ on where a write lands first. Forget the five separate boxes. Building a cache boils down to two brutal questions: who fetches the data when it's missing, your app or the cache layer itself? And where do writes land first: the database alone, both places, or the cache?

Cache-aside. Here your application does the heavy lifting: check Redis, and on empty, go pull the row from the DB and stuff it into the cache yourself. Pick it when reads dominate and a little staleness hurts nobody, because the cache stays optional. If Redis dies you get slow, not down. Most "we use Redis" setups are secretly this.

Read-through. Same read flow, except now it's the cache layer doing the fetching, not your app. It only pays off when your architecture already forces everything through a single chokepoint, a GraphQL federation layer, say, or a dedicated data-access service, because then the miss-handling code lives in one place instead of being copy-pasted across twelve services. Nobody mentions the bill, though. Your shiny new caching layer just became a single point of failure for every read you serve.

Write-through. You write to both at once, cache and DB, and in exchange a read straight after a write comes back fresh. The catch? You pay a double latency tax on every single write. Pay it only if the business demands instant read-after-write consistency. You'll also cache plenty of things nobody ever reads.

Write-back. This is vanity-metric territory. Page views. Like counts. Writes land in the cache and flush to the DB later, in batches, so they get fast and a crash loses whatever hadn't flushed yet. If "billing" and "write-back" show up in the same sentence, stop the meeting. A crash will silently delete money.

Write-around. Writes go straight to the DB; the cache only fills when something is read. Prevents write-heavy, read-never garbage from choking your memory limits, and pairs naturally with cache-aside. The one guarantee it makes in exchange: the first read after every write is a miss, which means your P99 read latency will spike after every write burst.

Know that going in.

Cache-aside plus write-around is the boring default. You need a stated reason to leave it, and "the diagram looked elegant" is not one.

Eviction: What Redis Actually Does When It's Full

Half the blog posts, and worse, half the production configs, assume Redis defaults to LRU (evict the least recently used key). It doesn't. By default, Redis ships with noeviction enabled, and with no maxmemory set it just grows until the kernel's OOM killer wakes up and shoots the entire process. Not failed writes. The whole node, offline.

Leave it on noeviction but cap the memory, and you've built a time bomb: writes start failing with OOM errors while reads keep working. Your cache is now a read-only brick, and on-call finds out from a wall of write errors instead of a memory graph.

redis.conf: the two lines most people never set

maxmemory 4gb

maxmemory-policy allkeys-lru   # default is noeviction: writes FAIL when full

Redis LRU is approximate. It samples N keys (default 5) and evicts

the best candidate. Raise it for closer-to-true LRU, at CPU cost.

maxmemory-samples 10

Keep it simple: LRU works flawlessly, until a dumb midnight scraper script walks 10,000 cold records and silently evicts your entire hot set (the small group of keys serving most of your traffic) in sixty seconds. LFU (Redis 4.0+, allkeys-lfu) evicts by how often a key is used instead, so it survives scans; the price is that it adapts slower when what's hot changes. TTL expires things on a clock. Eviction is what happens when memory runs out.

Pick allkeys-lru and move on. If you know exactly which nightly batch jobs will pollute your cache, switch to LFU instead. And one hard warning: don't mix persistent data and cache in one Redis. The volatile-* policies only ever evict keys that carry an expiration, which is the seatbelt that protects your real data when someone mixes them anyway; volatile-ttl just picks whichever expiring key dies soonest.

Invalidation Is a Ladder, Not a Debate

Stop treating invalidation like an engineering puzzle. It's a financial calculation: put a dollar amount on what a customer seeing old data costs you. You start at the bottom of the ladder where staleness is cheap, and you climb only when that number forces you. If it's zero, do nothing.

Rung 1: TTL only. A stale product blurb or avatar costs approximately nothing here. The TTL is your staleness budget said out loud, and "descriptions may be ten minutes old" is a business decision, so write it down where product can see it. Most caches should stop here and most teams can't accept that.

Rung 2: TTL plus delete-on-write. Stale data here is a UI embarrassment, not a fatal system error. Follower counts and dashboards live on this rung. The app deletes the key after saving the write; the next read refills the cache. Keep the TTL anyway, because the classic hole runs on a clock: a slow reader pulls the soon-to-be-stale row from the DB, your writer commits and deletes the key, and then the slow reader fills the now-empty key with its zombie copy. The TTL is what kills that zombie, capping the damage at minutes instead of forever.

Rung 3: A stale read is wrong behavior. Now you're into write-through with versioned keys (every write carries a version, and anything older than what's stored gets rejected, which kills the slow-reader race), or the answer nobody offers in design review: don't cache that read. Prices and permissions live here. Inventory lives here too. Oversell a warehouse count because of a stale cache, and you'll be paying refunds out of your own pocket. If the query is cheap and it absolutely must be right, stop putting it in Redis. Just hit Postgres.

If the team can't put the staleness budget in seconds, nobody chose the invalidation strategy. The defaults did.

The Three Ways a Cache Kills a Database

Avalanche. Stampede. Penetration. Strip the buzzwords off and they all mean the same terrifying thing: your database is about to eat the wall of traffic the cache was quietly absorbing.

Avalanche: everything expires at once. That's our warm-up story. The script filled a few hundred thousand keys in a loop and every SETEX got the same 3600 seconds. It was a random Tuesday and the deploy went completely green. Sixty minutes later every dashboard lit up red as the database CPU pinned at 100%. We blamed the deploy first, obviously; someone had a rollback half-typed before the graphs said otherwise. The app was fine. The database was drowning in reads it hadn't handled since before the cache existed.

A quick redis-cli TTL check told the real story: thousands of keys, all spawned by the same loop, all expiring within seconds of each other. The app's pool of database connections ran dry, timeouts spread into two services that don't even use that cache, and the database sat at full CPU for about forty minutes while we re-warmed with jitter and apologized in the incident channel.

The bug: identical TTLs create a synchronized expiry wave

r.setex(f"user:{uid}", 3600, blob)      # every key dies at t+1h, together

The fix costs one line: jitter (randomize) the TTL

ttl = 3600 + random.randint(0, 600)     # spread expiry across 10 minutes

r.setex(f"user:{uid}", ttl, blob)

Before you blindly copy-paste that snippet: say your database needs five minutes to rebuild 10,000 keys. A jitter window narrower than that hasn't prevented the avalanche, it's only scheduled it for ten minutes later.

Stampede: your most popular key dies. Within a millisecond 5,000 parallel requests notice, and all of them fire the same agonizing query together, an inside-job DDoS against your own infrastructure, then they trample each other writing the result back. The fix is request coalescing: one caller rebuilds, everyone else waits a beat or takes a stale copy.

Coalesce: only one worker recomputes the missing hot key

got_lock = r.set(f"lock:{key}", "1", nx=True, ex=10)

if got_lock:

val = expensive_query()             # one DB hit instead of 5,000

r.setex(key, ttl_with_jitter(), val)

else:

val = serve_stale_or_wait(key)      # stale beats a dead database

Warning: needs soft expiry tracked inside the payload (see below)

The shield in that snippet is nx=True, Redis's set-if-not-exists. It works as a cheap mutex: exactly one worker wins the lock and rebuilds, everyone else backs off. If you write Go, singleflight gives you this protection out of the box. There's also a catch the snippet hides. If the key truly expired, there's nothing stale left to serve, it's gone. Serving stale requires soft expiry. You embed an "expired_at" timestamp inside the payload itself but give the actual Redis key a much longer TTL, so an old copy stays around for everyone while one worker refreshes.

Penetration: requests for keys that don't exist. Deleted users, scraper garbage, someone trying IDs one by one. They will miss 100% of the time, punching straight through the cache and hammering Postgres directly. Cache the "not found" result for 30 to 60 seconds, and delete that marker the moment the real thing gets created, or new users spend their first minutes locked out of an account that exists. Put a bloom filter in front of the cache to reject impossible IDs before they ever touch Redis, if you can list out all valid IDs. And slap a hard rate limit on it anyway, because a predictable keyspace is an open invitation to scrapers who will happily DDoS your database for free.

The postmortem action item from our avalanche wasn't "tune Redis." It was "the database must survive a cold cache." We load-tested at a 0% hit rate and bought the missing headroom. Nobody enjoyed that invoice.

The Flow on a Sticky Note

Compressed to margin size:

The Interviewer Ladder

Interviewers aren't as creative as they think. They run the same script. Here's how to disarm their traps.

• "Where would you add a cache?" A named strategy plus the read/write ratio that justifies it.
• "How do you keep it consistent?" This one is fishing for the ladder: staleness budget first, then TTL vs delete-on-write vs write-through.
• "The cache restarts cold. Now what?" Avalanche and stampede, one fix for each, and proof the DB survives a 0% hit rate. The sticky note above is the answer key.
• Staff-level: "What would you refuse to cache?" Anything that lands on rung 3.

Mention invalidation before anyone asks; that's most of the interview right there. The anti-signal is "we'd use write-back for accuracy," usually a sign the phrase came from a listicle rather than an incident.

Architecture Review Defenses

If you survive the whiteboard, these are the three nitpicks that come next.

Redis or Memcached? Redis, unless raw multithreaded GET/SET speed on plain strings is the entire job. Memcached still wins that one niche. For data structures, replication, or persistence, it's Redis.

What's a good hit rate? Hit rates are vanity metrics. Database load is sanity. An 80% hit rate on a brutal 200ms join is worth far more than a 99% hit rate on a 2ms primary-key lookup. Watch DB load with the cache on versus off; that difference is what pays for the cluster.

In-process cache or Redis? Usually both, a short-TTL in-memory cache in each app node (L1) in front of a shared Redis (L2). But every local cache multiplies invalidation by your node count, and invalidating an L1 across 50 nodes takes a pub/sub broadcast, which is its own pile of architectural complexity. The L1 also lives inside your app's runtime heap (JVM/V8), so a fat L1 cache is a great way to buy longer garbage-collection pauses. Keep L1 TTLs under a minute and keep rung 3 data out.

The Part That Still Bites

Your cache will be empty at the worst possible moment, right after a failover or at the top of the biggest traffic day of the year. It goes cold exactly when the database can least afford it.

If your architecture collapses the moment Redis is empty, Redis isn't an optimization anymore. It's a load-bearing wall made of glass. Put the 0%-hit-rate load test on the calendar as a quarterly game day, because the real cold-cache day is on the calendar too. It just hasn't told you the date.

Thank you for reading. If you found this useful, follow PracHub for more practical deep dives on engineering, system design, and interview preparation written to help you build skills that last beyond a single article.