Design an Experimentation Platform: Experiment Data Model and Variant Assignment Logic

Design an A/B testing system that product teams use to run experiments. Go deep on two areas in particular: the data model behind experiments and their results, and the logic that splits users into variants. ```hint Follow one user through one test Trace what happens when a user reaches a feature under test: what decides their variant, what gets recorded, and how that record later becomes a result. ``` ### Clarifying Questions - Will experiments run on web, mobile and backend services, and must a user see the same variant on every device? - Must logged-out visitors be assigned too, or only signed-in users? - How many experiments run at the same time, and can one user be in several at once? - How fresh must results be: a daily report, or near-real-time monitoring that can stop a harmful test? ### What a Strong Answer Covers - Scope and requirements across the experiment lifecycle, from setup to decision - A data model for experiments, variants, targeting, exposures and metrics that supports trustworthy analysis - Assignment logic that is consistent, unbiased, fast, and safe when allocations change or experiments overlap - The path from logged events to metrics and statistical results, with data-quality checks - Scale, failure handling, observability and guardrails against harmful tests ### Follow-up Questions - The logs show noticeably more users in one arm of a test configured as 50/50. What do you check, and do you trust the results? - How would you let several teams run experiments on the same checkout page without their tests interfering? - How would you ramp a feature from a small share of users to everyone without users switching variants along the way?