System Design: The Rate Limiter

A rate limiter decides which requests may enter a service under a defined policy. This chapter connects the policy’s identity, time window, burst allowance, and rejection behavior to a distributed implementation.

What is a rate limiter?

A rate limiter bounds admitted work over time. For a hypothetical API allowing 500 requests per minute per client, define how a client is identified and whether “per minute” means a fixed window, a sliding window, or a refill rate with burst capacity. Those choices determine which requests exceed the policy.

Why do we need a rate limiter?

Rate limiting can reduce application-layer overload caused by mistakes or abusive traffic. It is one control among others: identity evasion, expensive individual requests, and traffic that saturates the network before reaching the limiter need additional defenses.