##### Question Design a high-throughput, distributed rate-limiting service that runs across multiple regions with low latency and burst tolerance. The service must enforce limits along several dimensions — per-user, per-API/route, the user×API combination, and an optional global cap — and approximate a sliding window. Target roughly 10M requests/second at peak. Cover the following: 1. **Algorithm choice.** Pick and justify an enforcement algorithm (e.g. token bucket vs. leaky bucket, and GCRA). Explain how you get burst tolerance and an approximately-sliding window, and how you compute `Retry-After`. 2. **API design.** Specify the on-request decision API (check-and-consume), the off-path budget/lease API, the policy (configuration) API, and an introspection/metrics API. 3. **Data model.** Define the policy schema and the per-key bucket state (rate, burst, tokens, last-refill timestamp, lease/epoch state). 4. **Architecture.** Describe how decisions are made on the request path with low latency — in particular how you avoid a remote/cross-region hop on every request (e.g. local in-process buckets fed by leased token budgets). 5. **Sharding and hot keys.** Partition the key space (consistent hashing with virtual nodes / bounded load) and mitigate hot keys (key splitting, dynamic split detection, single-writer fallback). 6. **Storage choices.** Compare in-memory vs. Redis vs. a custom store for enforcement counters, policy config, and telemetry. Justify why per-request Redis does not scale to 10M RPS. 7. **Clock and time-window semantics.** Handle refill timing, epoch boundaries, and clock skew across nodes and regions. 8. **Consistency trade-offs.** State what consistency you provide per-user/per-region vs. globally, and the bound on global overshoot. Show how to get strong consistency for a small subset of keys when required. 9. **Failures and partial outages.** Handle owner/coordinator unavailability, regional partitions, node failure, and duplicate requests (idempotency), including fail-open vs. fail-closed policy and refunds. 10. **Fairness.** Ensure fairness across users/tenants and, for global limits, across regions (e.g. demand-aware water-filling). 11. **Capacity planning and scaling.** Give the math for per-node throughput and machine count at baseline, how many machines to add for a 2× traffic spike, the autoscaling signals to watch, and backpressure/throttling strategy. 12. **Observability.** List the key metrics, traces, and alerts.

Pinterest software-engineer onsite system-design question: design a multi-region, high-throughput distributed rate limiter targeting ~10M req/s with per-user, per-API, and global limits, burst tolerance, and an approximately-sliding window. It tests token-bucket/GCRA algorithm choice, an architecture of local in-process buckets fed by epoch-based token leases, consistent-hashing sharding with hot-key splitting, bounded-overshoot consistency, fault handling, fairness, capacity-planning math (including a 2× spike), backpressure, and observability.

How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

What difficulty level is this interview question?

This is a hard difficulty System Design question, commonly asked during Onsite rounds at Pinterest.

What role is this question designed for?

This question is commonly asked for Software Engineer candidates at Pinterest during technical interviews.

Design a high-throughput distributed rate limiter

Question

Design a high-throughput, distributed rate-limiting service that runs across multiple regions with low latency and burst tolerance. The service must enforce limits along several dimensions — per-user, per-API/route, the user×API combination, and an optional global cap — and approximate a sliding window. Target roughly 10M requests/second at peak. Cover the following:

Algorithm choice. Pick and justify an enforcement algorithm (e.g. token bucket vs. leaky bucket, and GCRA). Explain how you get burst tolerance and an approximately-sliding window, and how you compute Retry-After .
API design. Specify the on-request decision API (check-and-consume), the off-path budget/lease API, the policy (configuration) API, and an introspection/metrics API.
Data model. Define the policy schema and the per-key bucket state (rate, burst, tokens, last-refill timestamp, lease/epoch state).
Architecture. Describe how decisions are made on the request path with low latency — in particular how you avoid a remote/cross-region hop on every request (e.g. local in-process buckets fed by leased token budgets).
Sharding and hot keys. Partition the key space (consistent hashing with virtual nodes / bounded load) and mitigate hot keys (key splitting, dynamic split detection, single-writer fallback).
Storage choices. Compare in-memory vs. Redis vs. a custom store for enforcement counters, policy config, and telemetry. Justify why per-request Redis does not scale to 10M RPS.
Clock and time-window semantics. Handle refill timing, epoch boundaries, and clock skew across nodes and regions.
Consistency trade-offs. State what consistency you provide per-user/per-region vs. globally, and the bound on global overshoot. Show how to get strong consistency for a small subset of keys when required.
Failures and partial outages. Handle owner/coordinator unavailability, regional partitions, node failure, and duplicate requests (idempotency), including fail-open vs. fail-closed policy and refunds.
Fairness. Ensure fairness across users/tenants and, for global limits, across regions (e.g. demand-aware water-filling).
Capacity planning and scaling. Give the math for per-node throughput and machine count at baseline, how many machines to add for a 2× traffic spike, the autoscaling signals to watch, and backpressure/throttling strategy.
Observability. List the key metrics, traces, and alerts.

Question

Algorithm choice. Pick and justify an enforcement algorithm (e.g. token bucket vs. leaky bucket, and GCRA). Explain how you get burst tolerance and an approximately-sliding window, and how you compute Retry-After .
API design. Specify the on-request decision API (check-and-consume), the off-path budget/lease API, the policy (configuration) API, and an introspection/metrics API.
Data model. Define the policy schema and the per-key bucket state (rate, burst, tokens, last-refill timestamp, lease/epoch state).
Architecture. Describe how decisions are made on the request path with low latency — in particular how you avoid a remote/cross-region hop on every request (e.g. local in-process buckets fed by leased token budgets).
Sharding and hot keys. Partition the key space (consistent hashing with virtual nodes / bounded load) and mitigate hot keys (key splitting, dynamic split detection, single-writer fallback).
Storage choices. Compare in-memory vs. Redis vs. a custom store for enforcement counters, policy config, and telemetry. Justify why per-request Redis does not scale to 10M RPS.
Clock and time-window semantics. Handle refill timing, epoch boundaries, and clock skew across nodes and regions.
Consistency trade-offs. State what consistency you provide per-user/per-region vs. globally, and the bound on global overshoot. Show how to get strong consistency for a small subset of keys when required.
Failures and partial outages. Handle owner/coordinator unavailability, regional partitions, node failure, and duplicate requests (idempotency), including fail-open vs. fail-closed policy and refunds.
Fairness. Ensure fairness across users/tenants and, for global limits, across regions (e.g. demand-aware water-filling).
Capacity planning and scaling. Give the math for per-node throughput and machine count at baseline, how many machines to add for a 2× traffic spike, the autoscaling signals to watch, and backpressure/throttling strategy.
Observability. List the key metrics, traces, and alerts.

Design a high-throughput distributed rate limiter

Quick Overview

Design a high-throughput distributed rate limiter

Question

Submit Your Answer to Earn 20XP

Design a high-throughput distributed rate limiter

Quick Overview

Design a high-throughput distributed rate limiter

Question

Submit Your Answer to Earn 20XP