Meta
Medium
Data ScientistDefine composite success for search and test it
Analytics & Experimentation
1
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Capital One
Hard
Data ScientistEvaluate merchant partnership for high-value customers
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistMeasure network effects and spillovers via experiments
Analytics & Experimentation
0
0
October 13, 2025
Meta
Medium
Data ScientistDiagnose a sudden KPI drop and validate causes
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistBuild dashboard; diagnose engagement–purchase gap
Analytics & Experimentation
1
0
October 13, 2025
Amazon
Hard
Data ScientistMeasure PMF for Alexa Shopping
Analytics & Experimentation
1
0
October 13, 2025
Thumbtack
Medium
Data ScientistExplain power drivers and resolve unexpected A/B results
Analytics & Experimentation
0
0
October 13, 2025
Other
Hard
Data ScientistDesign an A/B test for a Celebrate reaction
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistDesign experiment for unconnected content in feed
Analytics & Experimentation
1
0
October 13, 2025
Yahoo
Hard
Data ScientistDiagnose a 10% DAU drop
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistAnalyze Change in App Metrics and Feature Impact
Analytics & Experimentation
1
0
August 4, 2025
TikTok
Medium
Data ScientistDiagnose Traffic Allocation in A/B Test Results
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistDesign Metrics to Measure Inappropriate Content Severity and Prevalence
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistDesign an A/B Test for Group Video Calls Impact
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistDesign an Experiment to Evaluate New ML Model
Analytics & Experimentation
1
0
August 4, 2025
Meta
Medium
Data ScientistBoost Engagement and Purchases in Meta Social Products
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistDefine metrics for high-quality notifications
Analytics & Experimentation
1
0
July 27, 2025
Citi
Medium
Data ScientistHow would you A/B test first trade rate?
Analytics & Experimentation
1
0
March 25, 2025
Citi
Medium
Data ScientistHow would you analyze retail volume drop?
Analytics & Experimentation
1
0
March 25, 2025
Homedepot
Easy
Product AnalystDecide whether to keep a negative-margin promotion
Analytics & Experimentation
1
0
February 14, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.