Design and Implement a Rate Limiter with an AI Assistant, Then Defend and Test It

This onsite coding round allows AI assistants. You may bring your own laptop with any AI tools installed in advance, or use the interviewer's machine. Rounds of this kind come in two formats, refactor-and-debug or design-and-build. This one was design-and-build, and the task was to **design and implement a rate limiter**. The AI may write the code, but the design reasoning has to come from you. You must explain which algorithm you chose, why, which other algorithms exist and what the trade-offs are, and you must show, live, how you communicate with the AI, how you review the code it writes and how you test it. ### Constraints and Clarifications - Use any language you and your assistant work well in. - The interviewers judge your decisions and your process, not only whether the final code runs. ### Clarifying Questions - What is being limited, and per what key: requests per user, per API key, per IP address, or per endpoint? - What limits must it support, such as a number of requests per second or per minute, and must short bursts above the average rate be allowed? - What happens to a request over the limit: rejected immediately, queued, or delayed? Should the caller be told when to retry? - Does the limiter run inside one process, or must several servers enforce one shared limit? - How exact must the limit be at window boundaries, and is a small overshoot acceptable? ### Part 1 — Choose the algorithm and defend it Pick the rate-limiting algorithm you will implement. Explain why it fits the requirements you clarified, what the main alternatives are, and what each one trades off. ```hint Test each candidate at the boundary For each algorithm you consider, picture a client that sends its full quota just before a window boundary and again just after it. Ask what each algorithm lets through, and how much state it keeps per key. ``` #### What This Part Should Cover - The main algorithm families and how each behaves under bursts and at boundaries - The per-key memory and per-request cost of each - A choice justified against the clarified requirements, with the conditions under which you would choose differently ### Part 2 — Implement it with the AI assistant Have the assistant generate the implementation while you direct it. Show how you prompt it, and review what it produces before you accept it. ```hint Specify before you prompt Settle the interface, the time source and the concurrency expectations before asking for code, so you judge the output against a specification rather than against a first impression. ``` ```hint Read for the races When reviewing the generated code, look at every place where a counter or timestamp is read and then written back, and ask what another thread could do in between. ``` #### Clarifying Questions for this Part - Is it acceptable to use your own assistant configuration and prompt files, or should everything be typed during the session? - Should the limiter be a library class, or middleware in a small web service? #### What This Part Should Cover - Prompts that fix the interface, semantics and constraints instead of asking for "a rate limiter" - A review that finds real defects, such as races, wrong time handling or unbounded memory, and fixes them - Iterating with the assistant rather than accepting its first output ### Part 3 — Test it Show how you would test the limiter, and run the tests. ```hint Control the clock Tests that sleep are slow and flaky. Think about how the limiter could learn the time so that a test can move time forward exactly. ``` #### What This Part Should Cover - Deterministic tests at the exact refill or window boundaries - A concurrency test that would fail if the limiter could over-admit - Evidence that the tests, including any the assistant wrote, would actually catch a bug ### What a Strong Answer Covers - Requirements clarified before the algorithm is chosen - An algorithm comparison with concrete behavior at bursts and boundaries, not a list of names - A correct, thread-safe implementation with an injectable clock and bounded memory - Visible ownership of the design: the candidate directs the assistant, reads its code critically and can explain every line - Tests that pin the semantics, and a clear path from a single process to a shared, multi-server limit ### Follow-up Questions - Several servers must now share one limit per client. How do you keep the check-and-update atomic, and what is the latency cost? - The store behind a distributed limiter becomes unavailable. Should requests be allowed or rejected, and why? - Different customer tiers and endpoints need different limits that can change without a deploy. How does the design change? - How should a rejected client be told when to retry, and how do you keep clients from all retrying at the same instant?