Design an A/B testing platform
Company: Affirm
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Design an internal A/B testing platform for a consumer product company.
Product and engineering teams should be able to:
- create, update, pause, and archive experiments
- define control and treatment variants
- target experiments to specific user segments
- assign users consistently to variants across sessions
- log exposure, click, conversion, and custom events
- define primary metrics and guardrail metrics
- monitor experiment results in dashboards
- detect obvious data quality issues such as sample ratio mismatch
Your design should cover:
- APIs and core data model
- experiment configuration storage
- user bucketing and sticky assignment
- client/server SDKs for logging exposure and events
- event ingestion and storage
- metric computation and statistical analysis
- reliability, scalability, and low-latency requirements
- common pitfalls such as double counting, event loss, and invalid experiment reads
Quick Answer: This question evaluates the ability to design an end-to-end A/B testing platform, assessing competencies in distributed systems architecture, data modeling, event ingestion and storage, client/server SDKs, deterministic user bucketing, metric computation, and statistical analysis.