Design an In-Memory Key-Value Cache

Quick Overview

Design an in-memory key-value cache by fixing API, capacity, eviction, TTL, concurrency, and complexity semantics before coding.

Design an In-Memory Key-Value Cache

Company: OpenAI

Role: Machine Learning Engineer

Category: Software Engineering Fundamentals

Difficulty: hard

Interview Round: Technical Screen

# Design an In-Memory Key-Value Cache Design and implement an in-memory key-value cache. Because the source does not specify operations or policy, begin by clarifying the required API, capacity behavior, expiration, concurrency, and observability before selecting data structures. ### Constraints & Assumptions - At minimum, discuss get, put, update, and delete semantics. - Do not assume LRU eviction, TTL, persistence, or thread safety without stating it. - Keys and values fit in memory under the agreed capacity model. ### Clarifying Questions to Ask - Is capacity bounded by entries or bytes? - Is eviction required, and if so which policy? - Do entries expire, and can concurrent callers access the cache? ```hint Lock the contract first A cache data structure is only correct relative to explicit miss, overwrite, eviction, and expiry semantics. ``` ### What a Strong Answer Covers - API and exact return behavior. - Data structures matched to lookup and any clarified eviction policy. - Expiration, concurrency, memory accounting, and failure behavior. - Tests, complexity, metrics, and honest limits of an in-memory design. ### Follow-up Questions 1. How would you prevent a cache stampede? 2. What changes if values are mutable objects?

Quick Answer: Design an in-memory key-value cache by fixing API, capacity, eviction, TTL, concurrency, and complexity semantics before coding.

|Home/Software Engineering Fundamentals/OpenAI
OpenAI logo
OpenAI
Aug 23, 2026
hardMachine Learning EngineerTechnical ScreenSoftware Engineering Fundamentals
3
0

Design an In-Memory Key-Value Cache

Design and implement an in-memory key-value cache. Because the source does not specify operations or policy, begin by clarifying the required API, capacity behavior, expiration, concurrency, and observability before selecting data structures.

Constraints & Assumptions

  • At minimum, discuss get, put, update, and delete semantics.
  • Do not assume LRU eviction, TTL, persistence, or thread safety without stating it.
  • Keys and values fit in memory under the agreed capacity model.

Clarifying Questions to Ask Guidance

  • Is capacity bounded by entries or bytes?
  • Is eviction required, and if so which policy?
  • Do entries expire, and can concurrent callers access the cache?

What a Strong Answer Covers Guidance

  • API and exact return behavior.
  • Data structures matched to lookup and any clarified eviction policy.
  • Expiration, concurrency, memory accounting, and failure behavior.
  • Tests, complexity, metrics, and honest limits of an in-memory design.

Follow-up Questions Guidance

  1. How would you prevent a cache stampede?
  2. What changes if values are mutable objects?
Loading comments...