## Design a Smooth Per-Customer Calls-Per-Second Controller Design a service that controls outbound calls for many customers. Each customer has a configurable calls-per-second limit. Traffic should be spread smoothly rather than released as one burst each second. High-priority calls should be preferred over regular calls without starving regular work. One noisy customer must not delay others, and the service must remain available across failures. Include the request API, scheduling and pacing model, per-customer isolation, horizontal scaling, retries, timeouts, failure behavior, and operational trade-offs. ### Part 1 — Define Admission and Delivery Semantics Specify what a customer's CPS limit means, whether calls are accepted synchronously or queued, what acknowledgement represents, and the guarantees for ordering, duplication, cancellation, and priority. #### What This Part Should Cover - Separate start-rate, concurrency, and completion semantics. - Durable idempotent enqueue and observable request states. - Honest acknowledgement, cancellation, and duplicate-delivery guarantees. ```hint Separate acceptance from execution Persisting a request and actually starting its outbound call are different events with different failure modes. ``` ### Part 2 — Pace Calls Smoothly Design the per-customer scheduling state that spaces starts over time. Explain burst allowance, limit changes, clock behavior, high-priority selection, and a concrete anti-starvation policy for regular calls. #### What This Part Should Cover - A token bucket, virtual schedule, or equivalent precise pacing invariant. - Explicit burst capacity and controlled-clock behavior. - Priority preference with a measurable regular-traffic fairness bound. ```hint A one-second counter is not smooth pacing A token bucket or virtual next-send time can enforce a rate while controlling how closely starts cluster. ``` ### Part 3 — Partition and Recover Distribute many customers across workers while ensuring one active owner or equivalent atomic scheduler state per customer. Handle worker crash, reassignment, duplicated queue delivery, and customer-specific hot spots without creating a global lock. #### What This Part Should Cover - Stable customer partitioning and renewable ownership. - Fencing or atomic state transitions during failover. - Tenant isolation and a safe plan for exceptionally hot customers. ```hint Make ownership renewable A lease with an epoch lets another worker recover a customer while fencing a stale worker that later resumes. ``` ### Part 4 — Handle Call Outcomes and Operate the Service Define connect and response timeouts, retry eligibility, retry backoff, whether retries consume CPS capacity, dead-letter handling, overload behavior, metrics, and tests. #### What This Part Should Cover - Outcome classification, bounded backoff, and dead-letter policy. - Retries routed through the same rate-control path. - Backpressure, reconciliation, service-level metrics, and failure tests. ```hint Retry policy is part of rate policy An immediate retry can double outbound pressure unless it returns through the same pacing path. ``` ### What a Strong Answer Covers - A precise CPS and burst contract with smooth dispatch. - Durable per-customer queues, priority fairness, and tenant isolation. - Partition ownership, fencing, idempotency, and recovery after partial failure. - Backpressure and bounded work rather than unbounded memory growth. - Explicit retry, timeout, observability, and reconciliation behavior. ### Follow-up Questions 1. How would a customer's limit change take effect without creating a burst? 2. How would you support one customer whose rate exceeds one worker's capacity? 3. Which failure can still produce a duplicate outbound call, and where would idempotency live? 4. How would you prove that regular calls are not starved by a continuous priority backlog?