# Plan Eviction and Cleanup for a Production Cache A process-local timed cache is correct for basic puts and gets, but it can still exhaust memory, retain expired bookkeeping, or behave poorly under concurrency. Explain how you would turn it into a production-ready component. ### Constraints & Assumptions - Practice assumption: entries have heterogeneous sizes and TTLs, and the process has a hard memory budget. - Workload popularity can be skewed and can change over time. - Callers need bounded latency; a full-cache scan on an ordinary request is unacceptable. - The discussion should distinguish a single-process cache from a distributed cache service. ### Clarifying Questions to Ask - Is the limit a key count, an estimated byte budget, or both? - Does a stale entry need to disappear immediately, or only become unreadable immediately? - Are values recomputable, and is stale data ever acceptable? - What hit-rate, latency, and durability expectations matter? ### Part 1: Eviction and Out-of-Memory Control Compare suitable eviction policies, choose a default for the stated workload, and explain how you prevent the process from crossing its memory limit. #### Hints - Think about both admission and eviction. - Consider whether one unusually large value should displace many useful small values. #### What This Part Should Cover - A justified policy choice - Memory accounting and overload behavior - Workload-dependent trade-offs ### Part 2: Expiration Cleanup and Concurrency Describe lazy and proactive expiration cleanup, and explain how reads, writes, and cleanup coordinate without corrupting state. #### Hints - Immediate logical expiration and physical reclamation do not have to happen in the same operation. #### What This Part Should Cover - Bounded cleanup work - Synchronization boundaries - Race handling around expiration and replacement ### Part 3: Scale, Testing, and Operations Explain what changes if the cache becomes a shared service, and identify the tests and production signals you would require. #### Hints - Include failure behavior, not only the healthy path. #### What This Part Should Cover - Ownership or partitioning strategy - Correctness and load tests - Metrics that expose memory and hit-rate regressions ### What a Strong Answer Covers - Clear semantics before implementation choices - A policy tied to workload and memory constraints - Bounded request-path work and explicit concurrency control - Operational limits, degradation behavior, and measurable validation ### Follow-up Questions - How would you prevent many callers from recomputing the same expired key? - When would TinyLFU-style admission be preferable to plain LRU? - How would you roll out a new eviction policy without risking a fleet-wide miss storm?