Design a Shared A/B/C Experiment Assignment API for an IVR Phone Platform

Design a shared experiment-assignment API service for an interactive voice response (IVR) platform, the automated phone system that answers customer calls with menus and prompts. Given an experiment's configuration, the service returns which group, A, B or C, a caller or request belongs to, so that other services and applications can use it to launch features gradually and run experiments. The report states only this goal, so the traffic, the unit being assigned and the configuration format are open: agree on them with the interviewer before designing. ```hint Same caller, same group Think about how a caller who phones again tomorrow, or reaches a different server, lands in the same group without the service storing every assignment. ``` ```hint The call cannot wait An IVR flow decides what to play while the caller is on the line. Consider what the calling service should do if the assignment service is slow or unreachable. ``` ### Clarifying Questions - What is the unit of assignment: the caller's phone number, a customer account, or a single call? Must a returning caller always get the same group? - How many assignment requests per second arrive at peak, and what latency budget does one IVR step allow? - What does an experiment configuration contain: groups and their traffic shares, targeting rules, start and end times, a holdout? Who edits it, and how fast must a change take effect? - Can traffic shares change while an experiment runs, and must callers who are already assigned keep their group? - Must two experiments on the same IVR flow be mutually exclusive? - Must assignments be logged for analysis, and are phone numbers treated as sensitive data? ### What a Strong Answer Covers - Requirements, traffic and latency estimates, and the unit of assignment agreed first - An API and configuration model for experiments, groups, traffic shares and targeting - Deterministic, stateless assignment by hashing the unit with an experiment-specific salt, and how changing shares affects units already assigned - Fast, versioned and validated configuration distribution with an audit trail, and local evaluation in client libraries versus a central API - Exposure logging for analysis, privacy of caller identifiers, and safe defaults when the service is unavailable - Isolation between experiments, gradual rollout and a kill switch, scaling and monitoring ### Follow-up Questions - The treatment groups of a running experiment must grow without moving any caller who is already in a treatment group. How does your bucketing allow it? - Two teams run experiments on the same menu step. How do you keep them from interfering with each other? - The service is down during a traffic spike. What does each caller hear? - How would you detect that the observed split between groups differs from the configured one?