System Design: The Key-Value Store

A key-value store retrieves and updates data through a key. This chapter designs a distributed version, using hashing for placement, replication for additional copies, and versioning to handle concurrent updates.

Introduction to key-value stores

A key-value store associates a unique key with a value. The key is the application’s identifier; a hash may help decide where that key is stored. A distributed hash table (DHT) is one way to distribute such mappings, but not every key-value store is a DHT. Values can represent blobs, images, server names, or other application data, subject to the store’s supported types and size limits.

For this design, assume values are relatively small, roughly kilobytes to megabytes. Large objects can live in a blob store with a reference in the value field. Check the chosen implementation’s item limit and retrieval needs before adopting that split. Sessions are a common example of a workload accessed by a known key.