Requirements of a Rate Limiter’s Design

Requirements

Choose the unit of limited work, the identity that shares a quota, and the outcome when capacity is exhausted. These requirements determine both the counter model and where enforcement belongs.

Functional requirements

• Limit the number of requests a client can send within a time window.

• Ensure request limits per window are configurable.

• Notify the client (via error or notification) when a threshold is crossed.

Non-functional requirements

• Availability: Define which failures the limiter must tolerate and whether each endpoint rejects, allows, or uses a conservative local fallback when enforcement is unavailable.

• Latency: Set a budget for the decision, including identity lookup and counter access.

• Scalability: Size for request rate and active identity count, then bound stored state and coordination cost.

• Enforcement accuracy: State whether the limit is strict or may be exceeded by a bounded amount. A fast approximate check is a different contract from a strict quota.

Types of throttling

This course uses three policy categories. They describe how much excess work may be admitted, not the counting algorithm itself: