System Design: The Distributed Cache

A cache trades storage and freshness-management work for fewer expensive reads. This chapter designs a cache spread across servers and examines what happens when entries expire, nodes fail, or membership changes.

Problem statement

A typical system consists of three core components:

• A client requests a service.

• An application server handles the request.

• A database stores the service’s persistent data.

As traffic grows, repeated queries can become a database bottleneck. If many requests can reuse the same result for an acceptable period, a cache can reduce that repeated work. Measure the bottleneck first: not every slow request is solved by adding a cache.

A cache holds a reusable copy of data or a computed result. This design uses RAM for low-latency access, although caching as a general technique is not limited to memory.