System Design: In-Process Memory Usage Switcher

Context

You are designing an in-process memory guardrail for a backend service. The component monitors current memory consumption and toggles the application between modes to protect availability under memory pressure. It must operate on Linux and macOS, including containerized environments.

Objective

Design a memory usage switcher that monitors process memory and flips the application between NORMAL and DEGRADED modes when usage crosses configurable thresholds. Include the API, state logic (with hysteresis), runtime configurability, OS/container integration, multi-tenant budgeting, observability, reliability, and testing.

Functional Requirements

1. Modes and thresholds
   • Modes: NORMAL, DEGRADED.
   • Thresholds: warn at T_warn and shed at T_shed.
   • Hysteresis: distinct up/down thresholds to avoid flapping; debounce duration for sustained crossings.
2. Runtime configuration
   • Update thresholds at runtime.
   • Persist configuration with safe rollback.
3. API surface
   • setThresholds(config)
   • getState() → current mode, usage, effective limit, last transition reason/time.
   • registerCallbacks(hooks) → invoke on transitions (e.g., evict caches, slow batch jobs, reject requests).
4. Cross-platform inputs
   • Sampling process RSS.
   • OS memory pressure notifications.
   • Container/cgroup signals.
   • Compare trade-offs and define how to combine them.
5. Multi-tenant budgets
   • Components can register their own budgets/limits.
   • Enforce fair shedding under pressure.
6. Observability and operations
   • Emit metrics, logs, and traces.
   • Provide health/readiness checks.
   • Ensure safe shutdown behavior.
7. Non-functional
   • Concurrency and thread safety.
   • Failure modes and degradations.
   • Testing strategy: unit, load, fault injection.
   • Low overhead: sub-1% CPU, minimal syscall overhead.