Lyft
Medium
Data ScientistInvestigate Causes of Driver WOW Score Drop
Analytics & Experimentation
28
0
August 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Meta
Medium
Data ScientistEvaluate Impact of Targeting Ads to High-Intent Users
Analytics & Experimentation
52
0
July 12, 2025
DoorDash
Medium
Data ScientistDiagnose why average waiting time increased
Analytics & Experimentation
10
0
July 7, 2025
Netflix
Medium
Data ScientistCompute ITT, TOT, and LATE with noncompliance
Analytics & Experimentation
11
0
March 5, 2026
Meta
Medium
Data ScientistDiagnose Causes of Low Retention for FB Light
Analytics & Experimentation
36
0
August 4, 2025
OpenAI
Hard
Data ScientistDesign and analyze a free-trial A/B test
Analytics & Experimentation
11
0
October 13, 2025
Airbnb
Hard
Data ScientistAnalyze A/B test with rigorous diagnostics
Analytics & Experimentation
6
0
October 13, 2025
Meta
Medium
Data ScientistAnalyze User-Comment Distribution to Understand Engagement
Analytics & Experimentation
41
0
August 4, 2025
Meta
Easy
Data ScientistDesign metrics and experiment for stolen-post detection
Analytics & Experimentation
12
0
December 18, 2025
Lyft
Medium
Data ScientistDesign experiments for marketplace balance
Analytics & Experimentation
12
0
October 13, 2025
DoorDash
Hard
Data ScientistDrive app installs from web traffic
Analytics & Experimentation
7
0
October 13, 2025
Uber
Easy
Data ScientistDesign and Test a New Feature
Analytics & Experimentation
12
0
February 12, 2026
Meta
Easy
Data ScientistHow would you evaluate a new ads ranking algorithm?
Analytics & Experimentation
11
0
October 30, 2025
DoorDash
Hard
Data ScientistDecide and test a 20% discount strategy
Analytics & Experimentation
6
0
October 13, 2025
Capital One
Hard
Data ScientistEvaluate launching a vegan burger
Analytics & Experimentation
9
0
October 13, 2025
Anthropic
Hard
Software EngineerDesign a profiling plan for kernels
Analytics & Experimentation
10
0
September 6, 2025
Meta
Easy
Product AnalystDesign experiments and diagnose metric changes
Analytics & Experimentation
7
0
February 22, 2026
Roku
Medium
Data ScientistMeasure Speaker's Impact Using Propensity Score Matching
Analytics & Experimentation
36
0
July 12, 2025
Netflix
Easy
Data ScientistEstimate ATE, ITT, and TOT from experiment
Analytics & Experimentation
6
0
January 17, 2026
DoorDash
Hard
Data ScientistDiagnose Causes of High Out-of-Stock Rate in Groceries
Analytics & Experimentation
9
0
August 4, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.