DoorDash
Medium
Data ScientistDetermine Success Metrics for Biker Dasher Program Launch
Analytics & Experimentation
2
0
August 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Coinbase
Medium
Data ScientistInvestigate Sudden Revenue Drop: Steps and Metrics Analyzed
Analytics & Experimentation
2
0
August 4, 2025
TikTok
Medium
Data ScientistImprove TikTok's Algorithm for Diverse Content Discovery
Analytics & Experimentation
16
0
July 12, 2025
PayPal
Easy
Data ScientistHow to evaluate a new homepage feature
Analytics & Experimentation
3
0
February 15, 2026
Google
Medium
Data ScientistEstablish causality: commute playlist and driving speed
Analytics & Experimentation
1
0
October 13, 2025
Google
Medium
Data ScientistAssess education–income effect credibly
Analytics & Experimentation
4
0
October 13, 2025
Attentive
Hard
Data ScientistInvestigate SMS delivery-rate drop at Attentive
Analytics & Experimentation
2
0
October 13, 2025
Capital One
Hard
Data ScientistDesign A/B test for credit card offer
Analytics & Experimentation
3
0
October 13, 2025
Tools For Humanity
Easy
Data ScientistDetect and evaluate "stolen" posts
Analytics & Experimentation
3
0
August 29, 2025
Gusto
Medium
Data ScientistIdentify Growth Opportunities for New Payroll Feature Launch
Analytics & Experimentation
2
0
August 4, 2025
DoorDash
Medium
Data ScientistInvestigate Pop-up Impact on Partner Referral Conversions
Analytics & Experimentation
3
0
August 4, 2025
Apple
Medium
Data ScientistDesign an A/B Test for Homepage Layout Impact
Analytics & Experimentation
3
0
August 4, 2025
LinkedIn
Medium
Data ScientistPredict Impact of 'Online Indicator' Feature
Analytics & Experimentation
10
0
July 12, 2025
Uber
Medium
Data ScientistEvaluate UberEATS priority delivery and membership
Analytics & Experimentation
3
0
February 28, 2026
Uber
Medium
Data ScientistHow would you evaluate UberEats growth?
Analytics & Experimentation
4
0
January 22, 2026
Roku
Medium
Data ScientistMeasure speaker impact without A/B testing
Analytics & Experimentation
3
0
October 13, 2025
TikTok
Hard
Data ScientistDesign an interference-robust A/B test for monetization
Analytics & Experimentation
4
0
October 13, 2025
DoorDash
Medium
Data ScientistIdentify Key Metrics to Address Delivery Delays
Analytics & Experimentation
2
0
August 4, 2025
Meta
Easy
Data ScientistExpected impressions per user under random assignment
Analytics & Experimentation
12
0
July 12, 2025
Instacart
Easy
Data ScientistHow to debug an apparent D14 retention drop
Analytics & Experimentation
2
0
February 6, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.