Design a memory usage switcher with thresholds

Q: Design a memory usage switcher with thresholds

This question evaluates system design and operational engineering skills in process memory management, runtime configurability, observability, and reliability, and belongs to the System Design domain; it primarily tests practical application with system-level conceptual reasoning.

Q: How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty System Design question, commonly asked during Technical Screen rounds at Amazon.

Q: What role is this question designed for?

This question is commonly asked for Software Engineer candidates at Amazon during technical interviews.

Question

System Design: In-Process Memory Usage Switcher

Context

You are designing an in-process memory guardrail for a backend service. The component monitors current memory consumption and toggles the application between modes to protect availability under memory pressure. It must operate on Linux and macOS, including containerized environments.

Objective

Design a memory usage switcher that monitors process memory and flips the application between NORMAL and DEGRADED modes when usage crosses configurable thresholds. Include the API, state logic (with hysteresis), runtime configurability, OS/container integration, multi-tenant budgeting, observability, reliability, and testing.

Functional Requirements

Modes and thresholds
- Modes: NORMAL, DEGRADED.
- Thresholds: warn at T_warn and shed at T_shed.
- Hysteresis: distinct up/down thresholds to avoid flapping; debounce duration for sustained crossings.
Runtime configuration
- Update thresholds at runtime.
- Persist configuration with safe rollback.
API surface
- setThresholds(config)
- getState() → current mode, usage, effective limit, last transition reason/time.
- registerCallbacks(hooks) → invoke on transitions (e.g., evict caches, slow batch jobs, reject requests).
Cross-platform inputs
- Sampling process RSS.
- OS memory pressure notifications.
- Container/cgroup signals.
- Compare trade-offs and define how to combine them.
Multi-tenant budgets
- Components can register their own budgets/limits.
- Enforce fair shedding under pressure.
Observability and operations
- Emit metrics, logs, and traces.
- Provide health/readiness checks.
- Ensure safe shutdown behavior.
Non-functional
- Concurrency and thread safety.
- Failure modes and degradations.
- Testing strategy: unit, load, fault injection.
- Low overhead: sub-1% CPU, minimal syscall overhead.

Design a memory usage switcher with thresholds

Quick Overview

System Design: In-Process Memory Usage Switcher

Context

Objective

Functional Requirements

Solution

Comments (0)