Design scalable LRU cache and backtracking API

Quick Overview

Design scalable LRU cache and backtracking API evaluates requirements, scale assumptions, API/data design, architecture, trade-offs, failure modes, and rollout in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Design scalable LRU cache and backtracking API

System Design: Two Production-Ready Services

A) Thread-Safe LRU Caching Service

Design and describe a production-ready, in-process LRU cache that can also be deployed as a networked cache service.

Specify:

API surface (in-process library and optional network endpoints): get/put/delete/computeIfAbsent, per-entry TTL, size limits, stats.
Concurrency control inside a process (e.g., fine-grained locks, lock striping, or lock-free techniques). Ensure eviction is correct under concurrent access.
TTL and capacity management (entry count limit, weighted by bytes, admission control).
Metrics and observability (hits/misses, evictions, latencies, saturation, traces, logs, debug endpoints).
Scaling across multiple instances:
- Using Redis (or another remote cache): topology, client behavior, backpressure, failure handling.
- Or client-side sharding with consistent hashing: hot key mitigation, replication/failover, rebalancing, consistency model.
Handling hot keys, backpressure, and data consistency semantics (cache-aside/write-through, stale reads, idempotency of updates, CAS).

Provide concrete choices and trade-offs.

B) Compute-Heavy "Find Transfer Combinations" API

Design an API that computes transfer route combinations (e.g., finding valid paths across payment rails/providers with constraints), with potential traffic spikes.

Specify:

Stateless vs. stateful design; avoid shared mutable state in workers and define per-request memory management.
Caching and precomputation strategies; request de-duplication (in-process and cross-instance).
Rate limiting, timeouts/deadlines, cancellation, idempotency.
Horizontal scaling and autoscaling policies; synchronous vs. async job modes.
Capacity estimates with clear assumptions and formulas.
A testing plan covering correctness, performance, reliability, and operability.

Make any minimal, explicit assumptions needed so the design is concrete.

Clarifying Questions to Ask Guidance

Clarify users, core use cases, read/write patterns, scale, latency, availability, and data retention.
State explicit assumptions before making sizing or architecture decisions.
Prioritize the functional path first, then address reliability, security, observability, and rollout.

What a Strong Answer Covers Guidance

A scoped requirements summary with concrete non-goals and success metrics.
API, data model, architecture, consistency, capacity, and operations.
Reasoned trade-offs among simple and scalable designs, including bottlenecks and failure modes.
A validation, monitoring, migration, and launch plan appropriate for the risk level.

Follow-up Questions Guidance

What breaks first at 10x traffic or data volume?
How would you degrade gracefully during dependency failures?
What metrics and alerts would prove the design is healthy after launch?

Design scalable LRU cache and backtracking API

Quick Overview

Design scalable LRU cache and backtracking API

Design scalable LRU cache and backtracking API

System Design: Two Production-Ready Services

A) Thread-Safe LRU Caching Service

B) Compute-Heavy "Find Transfer Combinations" API

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Submit Your Answer to Earn 20XP