Design a memory usage switcher with thresholds

Quick Overview

This question evaluates system design and operational engineering skills in process memory management, runtime configurability, observability, and reliability, and belongs to the System Design domain; it primarily tests practical application with system-level conceptual reasoning.

Design a memory usage switcher with thresholds

Company: Amazon

Role: Software Engineer

Category: System Design

Difficulty: hard

Interview Round: Technical Screen

Design an in-process memory usage switcher that monitors current memory consumption and flips the application between NORMAL and DEGRADED modes when usage crosses configurable thresholds. Requirements: support runtime updates to thresholds (e.g., warn at T_warn, shed at T_shed) with hysteresis to prevent flapping; expose APIs (setThresholds, getState, registerCallbacks) and trigger actions on transitions (evict caches, slow batch jobs, reject requests); work across Linux and macOS—compare sampling process RSS, OS memory-pressure notifications, and container/cgroup signals; support multi-tenant budgets where components register their own limits; emit metrics and logs, provide health checks, and ensure safe shutdown; address concurrency and thread safety, failure modes, testing strategy (load tests, fault injection), and configuration persistence and rollback.

Overview: This question evaluates system design and operational engineering skills in process memory management, runtime configurability, observability, and reliability, and belongs to the System Design domain; it primarily tests practical application with system-level conceptual reasoning.

|Home/System Design/Amazon
Amazon logo
Amazon
Sep 6, 2025
hardSoftware EngineerTechnical ScreenSystem Design
8
0

System Design: In-Process Memory Usage Switcher

Context

You are designing an in-process memory guardrail for a backend service. The component monitors current memory consumption and toggles the application between modes to protect availability under memory pressure. It must operate on Linux and macOS, including containerized environments.

Objective

Design a memory usage switcher that monitors process memory and flips the application between NORMAL and DEGRADED modes when usage crosses configurable thresholds. Include the API, state logic (with hysteresis), runtime configurability, OS/container integration, multi-tenant budgeting, observability, reliability, and testing.

Functional Requirements

  1. Modes and thresholds
    • Modes: NORMAL, DEGRADED.
    • Thresholds: warn at T_warn and shed at T_shed.
    • Hysteresis: distinct up/down thresholds to avoid flapping; debounce duration for sustained crossings.
  2. Runtime configuration
    • Update thresholds at runtime.
    • Persist configuration with safe rollback.
  3. API surface
    • setThresholds(config)
    • getState() → current mode, usage, effective limit, last transition reason/time.
    • registerCallbacks(hooks) → invoke on transitions (e.g., evict caches, slow batch jobs, reject requests).
  4. Cross-platform inputs
    • Sampling process RSS.
    • OS memory pressure notifications.
    • Container/cgroup signals.
    • Compare trade-offs and define how to combine them.
  5. Multi-tenant budgets
    • Components can register their own budgets/limits.
    • Enforce fair shedding under pressure.
  6. Observability and operations
    • Emit metrics, logs, and traces.
    • Provide health/readiness checks.
    • Ensure safe shutdown behavior.
  7. Non-functional
    • Concurrency and thread safety.
    • Failure modes and degradations.
    • Testing strategy: unit, load, fault injection.
    • Low overhead: sub-1% CPU, minimal syscall overhead.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...