Find the Production Bugs in a Dictionary Cache Around an Expensive Computation

The interviewer shows you a short piece of code that caches the results of an expensive function, and asks you to find the problems with it. The original snippet is not preserved. The version below reconstructs the same pattern, an in-process dictionary cache in front of a function named `expensive_computation`, shown here in Python, and it is the code you should review. Assume it runs inside a long-lived, multi-threaded web service in which many requests call `get_result` concurrently. ```python _cache = {} def get_result(key): if key in _cache: return _cache[key] result = expensive_computation(key) _cache[key] = result return result ``` List every problem you can find, explain why each one matters in production, and propose a fix. ```hint Run it for a month Picture this function after weeks inside a busy service that sees many distinct keys. What happens to `_cache`? ``` ```hint Mind the gap Walk through what each concurrent caller does between the membership check and the assignment, and how long that gap can last. ``` ```hint Beyond the happy path Ask what `expensive_computation` can do other than return a good value, and what kinds of `key` callers might pass in. ``` ### Constraints and Clarifications - `expensive_computation` is slow compared with the rest of the request, and it can fail. - You may change the function's internals, but callers must keep calling `get_result(key)`. ### Clarifying Questions - How many distinct keys exist, and how large is each result? - Can the result for a given key change over time, so that cached entries need a time-to-live? - Does `expensive_computation` signal failure by raising an exception, by returning an error value, or both? Are its failures transient? - Is the service one process with many threads, several processes, or several machines? ### What a Strong Answer Covers - Unbounded memory growth, and the need for a size limit together with an eviction policy - Concurrent misses for the same key all running the expensive computation, and how to collapse them into one - Behavior when the computation fails: what is and is not cached, how failures reach callers, and when to retry - Validation of the key before it is used as a cache key or passed to the computation - Correctness of the fix itself under concurrency, and the metrics that show whether the cache is working ### Follow-up Questions - Implement a bounded least-recently-used version and state the time complexity of each operation. - How would you share this cache across several service instances, and what new failure modes appear? - When is caching a failure (negative caching) the right choice, and for how long?