Detailed Design of a Distributed Cache

The initial cache needs a membership protocol, replicas for selected failures, and a concrete lookup/eviction implementation. Add these while preserving clear rules for who owns a key and which version a reader may see.

Find and remove limitations

Before we get to the detailed design, we must resolve three specific challenges:

• Service discovery: Cache clients have no mechanism to detect when cache servers are added or fail.

• SPOF and performance: Using a single server for a dataset creates a Single Point of Failure (SPOF). Additionally, frequently accessed data (hotkeys) can overload a single node, degrading performance.

• Server internals: The design lacks details regarding internal data structures and eviction policies.

Maintain the cache servers list

We will address the service discovery problem first. The following slides illustrate three potential strategies: