Design an In-Memory Key-Value Cache
Company: OpenAI
Role: Machine Learning Engineer
Category: Software Engineering Fundamentals
Difficulty: hard
Interview Round: Technical Screen
# Design an In-Memory Key-Value Cache
Design and implement an in-memory key-value cache. Because the source does not specify operations or policy, begin by clarifying the required API, capacity behavior, expiration, concurrency, and observability before selecting data structures.
### Constraints & Assumptions
- At minimum, discuss get, put, update, and delete semantics.
- Do not assume LRU eviction, TTL, persistence, or thread safety without stating it.
- Keys and values fit in memory under the agreed capacity model.
### Clarifying Questions to Ask
- Is capacity bounded by entries or bytes?
- Is eviction required, and if so which policy?
- Do entries expire, and can concurrent callers access the cache?
```hint Lock the contract first
A cache data structure is only correct relative to explicit miss, overwrite, eviction, and expiry semantics.
```
### What a Strong Answer Covers
- API and exact return behavior.
- Data structures matched to lookup and any clarified eviction policy.
- Expiration, concurrency, memory accounting, and failure behavior.
- Tests, complexity, metrics, and honest limits of an in-memory design.
### Follow-up Questions
1. How would you prevent a cache stampede?
2. What changes if values are mutable objects?
Quick Answer: Design an in-memory key-value cache by fixing API, capacity, eviction, TTL, concurrency, and complexity semantics before coding.