Apple
Medium
Data ScientistDiagnose post-release conversion regression rigorously
Analytics & Experimentation
4
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Google
Medium
Data ScientistEstimate sales impact from reviews causally
Analytics & Experimentation
3
0
October 13, 2025
Coinbase
Hard
Data ScientistDiagnose a 20% retail revenue drop
Analytics & Experimentation
3
0
October 13, 2025
Meta
Hard
Data ScientistDefine metrics for harmful-content severity
Analytics & Experimentation
3
0
September 19, 2025
Meta
Hard
Data ScientistInvestigate Reasons for Higher Instagram Story Consumption
Analytics & Experimentation
13
0
July 12, 2025
Meta
Medium
Data ScientistDesign marketplace experiments at DoorDash
Analytics & Experimentation
2
0
March 11, 2026
OpenAI
Hard
Data ScientistMeasure free-month promotion impact
Analytics & Experimentation
3
0
January 15, 2026
Capital One
Hard
Data ScientistOptimize amusement park pricing, capacity, and testing
Analytics & Experimentation
3
0
October 13, 2025
Uber
Hard
Data ScientistDesign an experiment with marketplace network effects
Analytics & Experimentation
3
0
October 13, 2025
Airbnb
Hard
Data ScientistDesign and assess an A/B test
Analytics & Experimentation
2
0
September 6, 2025
DoorDash
Medium
Data ScientistAnalyze Causes for 10% Decline in LA Deliveries
Analytics & Experimentation
4
0
August 4, 2025
PayPal
Medium
Data ScientistExplain P-Value and Errors in A/B Testing
Analytics & Experimentation
3
0
August 4, 2025
OneMain Financial
Easy
Data ScientistCalculate Break-even for New Credit Card Product Launch
Analytics & Experimentation
4
0
August 4, 2025
DoorDash
Medium
Data ScientistEvaluate Key Metrics for Biker-Dasher Program Success
Analytics & Experimentation
3
0
August 4, 2025
Capital One
Medium
Data ScientistEvaluate a credit-card acquisition partnership
Analytics & Experimentation
4
0
October 13, 2025
TikTok
Hard
Data ScientistDiagnose metric drop in Ads Manager
Analytics & Experimentation
4
0
October 13, 2025
Amazon
Hard
Data ScientistDesign causal study for reminder impact
Analytics & Experimentation
3
0
October 13, 2025
Apple
Easy
Data ScientistInvestigate cross-country engagement and ads experiments
Analytics & Experimentation
3
0
August 24, 2025
Shopify
Medium
Data ScientistRecommend Next Steps for Pirate Theme Optimization
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistEvaluate Facebook Groups Metrics and Test Comment-Collapsing Feature
Analytics & Experimentation
12
0
August 4, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.