Google
Hard
Data ScientistDiagnose unbiasedness in a messy A/B test
Analytics & Experimentation
1
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Google
Hard
Data ScientistChoose a precise A/B test primary metric
Analytics & Experimentation
2
0
October 13, 2025
Meta
Medium
Data ScientistDecide event notification launch via experiments
Analytics & Experimentation
1
0
October 13, 2025
Coinbase
Hard
Data ScientistEstimate Super Bowl QR-driven registrations
Analytics & Experimentation
2
0
October 13, 2025
TikTok
Hard
Data ScientistDesign robust A/B test with interference and seasonality
Analytics & Experimentation
2
0
October 13, 2025
TikTok
Hard
Data ScientistDrive product decisions with causal product sense
Analytics & Experimentation
2
0
October 13, 2025
Meta
Easy
Data ScientistDetermine if users need a new feature
Analytics & Experimentation
1
0
October 11, 2025
Meta
Hard
Data ScientistEstimate Fake Accounts Using Data Signals and Sampling
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistCompare Instagram and Facebook Stories Using Key Performance Metrics
Analytics & Experimentation
1
0
August 4, 2025
Meta
Medium
Data ScientistEvaluate Success of 'Similar Listings' Notification Feature
Analytics & Experimentation
3
0
August 4, 2025
Meta
Hard
Data ScientistEvaluate New Ad Model with A/B Testing Experiment
Analytics & Experimentation
1
0
August 4, 2025
Microsoft
Medium
Data ScientistDesign Testing Without A/B Experiments
Analytics & Experimentation
1
0
February 25, 2026
Coinbase
Medium
Data ScientistDesign Identity-Trust A/B Test
Analytics & Experimentation
2
0
February 13, 2026
Instacart
Easy
Data ScientistShould you roll out if NSM decreases?
Analytics & Experimentation
1
0
February 6, 2026
Uber
Medium
Data ScientistHow should Uber evaluate lower ETA?
Analytics & Experimentation
3
0
January 30, 2026
Roblox
Medium
Data ScientistHow to estimate a feature’s causal impact on time spent
Analytics & Experimentation
1
0
January 13, 2026
Amazon
Easy
Data ScientistHow to evaluate adding video ads in a game
Analytics & Experimentation
1
0
December 9, 2025
Roblox
Hard
Data ScientistCompute DID estimate and pretrend flag
Analytics & Experimentation
2
0
November 24, 2025
Roku
Medium
Data ScientistDiagnose coffee-shop profit decline
Analytics & Experimentation
2
0
October 13, 2025
Meta
Medium
Data ScientistDesign a study to compare social vs game engagement
Analytics & Experimentation
2
0
October 13, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.