Meta
Hard
Data ScientistEstimate revenue of organic shopping tab
Analytics & Experimentation
2
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Meta
Hard
Data ScientistDesign experiment with network and novelty effects
Analytics & Experimentation
2
0
October 13, 2025
Meta
Hard
Data ScientistSelect interest thresholds under skewness and cost
Analytics & Experimentation
2
0
October 13, 2025
CVS Health
Hard
Data ScientistDesign a flu-shot A/B/n campaign experiment
Analytics & Experimentation
2
0
October 13, 2025
Amazon
Hard
Data ScientistProve new allocation outperforms manual baseline
Analytics & Experimentation
2
0
October 13, 2025
TikTok
Hard
Data ScientistCausally measure traffic reduction effectiveness
Analytics & Experimentation
1
0
October 13, 2025
Roblox
Hard
Data ScientistDesign and analyze an ads ranking experiment
Analytics & Experimentation
1
0
October 13, 2025
Capital One
Hard
Data ScientistChallenge and validate assumptions
Analytics & Experimentation
2
0
October 13, 2025
Capital One
Hard
Data ScientistDesign and analyze a card signup A/B test
Analytics & Experimentation
1
0
October 13, 2025
DoorDash
Hard
Analytics EngineerAllocate Support Cost and Diagnose Decline
Analytics & Experimentation
1
0
October 12, 2025
LinkedIn
Medium
Data ScientistHow to analyze Simpson's paradox
Analytics & Experimentation
1
0
September 5, 2025
Meta
Hard
Data ScientistDetermine User Demand for New Video-Calling Feature
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistEvaluate Instagram Shopping Tab Success with Key Metrics
Analytics & Experimentation
1
0
August 4, 2025
Meta
Medium
Data ScientistAnalyze Comment Distribution Using Statistical Metrics and Tests
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistConvince Leadership to Launch Group Chat Feature
Analytics & Experimentation
2
0
August 4, 2025
BetterHelp
Hard
Data ScientistOptimize Experiment Thresholds for Impactful Feature Launches
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistEvaluate Success of B2C Chat App with Key Metrics
Analytics & Experimentation
3
0
July 12, 2025
Meta
Medium
Product AnalystAnalyze DoorDash marketplace product decisions
Analytics & Experimentation
1
0
February 19, 2026
Coinbase
Medium
Data ScientistDesign Identity & Trust Experiment
Analytics & Experimentation
1
0
January 21, 2026
Uber
Medium
Data ScientistEvaluate business value of lower ETA
Analytics & Experimentation
2
0
January 18, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.