Design a distributed rate limiter

Q: Design a distributed rate limiter

This question evaluates skills in distributed systems design, concurrency control, rate-limiting algorithms (token bucket and sliding window), distributed coordination and consistency, API design, and operational monitoring for scalability and fault tolerance.

Q: How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

Question

Design a Rate Limiter with Burst Allowance and Distributed Coordination

Context

You are designing a rate limiter for an API gateway that serves high QPS traffic. The limiter should cap requests to a configured rate (QPS) with an optional burst allowance, operate correctly under concurrency, and scale across multiple application instances.

Requirements

Single-instance design:
- Implement and compare two approaches: token bucket and sliding window.
- Specify data structures, algorithms, and precise behavior (including burst allowance).
- Provide time/space complexity.
- Ensure correctness under concurrent access.
Distributed design across multiple app instances:
- Coordination strategies: centralized store (e.g., Redis) vs. sharded counters.
- Address clock skew tolerance, atomicity, idempotency.
- Discuss failure modes (node loss, partial updates) and strategies for consistency and fairness.
Developer-facing API:
- Define clear APIs (e.g., Allow, Acquire) and example configurations.
Operations:
- Monitoring/alerting for saturation and error conditions.

Provide code-like pseudocode where helpful.

Design a distributed rate limiter

Design a Rate Limiter with Burst Allowance and Distributed Coordination

Context

Requirements

Solution

Comments (0)

Design a distributed rate limiter

Overview

Design a Rate Limiter with Burst Allowance and Distributed Coordination

Context

Requirements

Solution

Comments (0)