Design an in-memory key-value database

Q: Design an in-memory key-value database

This is a System Design interview question from OpenAI for Software Engineer roles. View the full question and solution on PracHub.

Q: How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

Question

Design an In-Memory Key–Value Database (Technical Screen)

Context

Build an in-memory key–value (KV) database that offers high-throughput, low-latency operations. It should keep most data in RAM, support optional persistence, and scale across CPU cores and machines.

Assume a typical deployment runs on commodity servers with SSDs and multiple CPU cores. The system must handle high QPS with O(1) average-time reads and writes.

Functional Requirements

Core operations:
- set(key, value[, ttl])
- get(key)
- delete(key)
- compare-and-set (CAS): conditional update based on version or expected value
TTL expiration for keys
Atomic multi-key operations (within a shard)

Non-Functional Requirements

O(1) average-time reads/writes
Expiration tracking (e.g., timing wheel or min-heap)
Memory management and eviction policy (e.g., LRU)
Durability via append-only log (AOF) with periodic snapshots
Concurrency control strategy (single-threaded event loop vs locks)
Crash recovery procedure
Replication and sharding for scale
Consistency guarantees
Backpressure under overload
Observability: logs, metrics, tracing

Deliverables

API definitions (client-facing)
In-memory data structures and algorithms
On-disk record schemas (AOF and snapshot)
Concurrency and transaction model (incl. atomic multi-key)
Crash-recovery flow
Replication and sharding design
Consistency guarantees and trade-offs
Backpressure strategies
Failure modes and metrics