DoorDash
Medium
Data ScientistAnalyze Biker Dasher Program's Impact and Success Metrics
Analytics & Experimentation
5
0
August 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Shopify
Medium
Data ScientistAnalyze Trends to Optimize Pirate-Theme Product Strategy
Analytics & Experimentation
20
0
August 4, 2025
Meta
Medium
Data ScientistEvaluate an ads algorithm change
Analytics & Experimentation
6
0
February 22, 2026
Upstart
Hard
Data ScientistEvaluate channels and allocate budget
Analytics & Experimentation
7
0
October 13, 2025
TikTok
Medium
Data ScientistBalance Customer Satisfaction with Fraud Prevention: Key Metrics to Track
Analytics & Experimentation
22
0
July 12, 2025
Google
Medium
Data ScientistAnalyze Call Drop Rates Pre- and Post-Update Implementation
Analytics & Experimentation
19
0
July 12, 2025
Instacart
Medium
Data ScientistInvestigate Instacart Revenue Decline Using Weekly Data
Analytics & Experimentation
21
0
July 12, 2025
Google
Medium
Data ScientistAnalyze Impact of Customer Reviews on Sales Performance
Analytics & Experimentation
19
0
July 12, 2025
DoorDash
Hard
Data ScientistEvaluate Dasher Initiatives with A/B Testing and Metrics
Analytics & Experimentation
6
0
August 4, 2025
Meta
Medium
Data EngineerAnalyze private-account product metrics
Analytics & Experimentation
4
1
August 4, 2025
Google
Medium
Data ScientistEvaluate College Impact on Income: Address Bias and Validity
Analytics & Experimentation
17
0
July 12, 2025
SoFi
Hard
Data ScientistPlan and validate ranking experiment
Analytics & Experimentation
7
0
October 13, 2025
Netflix
Hard
Data ScientistDesign and power a frequency-cap experiment
Analytics & Experimentation
7
0
October 13, 2025
DoorDash
Medium
Data ScientistInvestigate Causes of Increased Driver Wait Time
Analytics & Experimentation
5
0
August 4, 2025
TikTok
Medium
Data ScientistTrack Key Metrics for Apple's New Phone Launch
Analytics & Experimentation
23
0
July 12, 2025
Meta
Medium
Data ScientistInterpreting confidence intervals to choose a treatment
Analytics & Experimentation
10
0
July 12, 2025
Yahoo
Medium
Data ScientistInvestigate Yahoo Mail's 10% DAU Decline Causes
Analytics & Experimentation
27
0
July 12, 2025
TikTok
Medium
Data ScientistAnalyze Negative Reviews' Impact on Coupon Repurchase Rate
Analytics & Experimentation
21
0
July 12, 2025
Lyft
Hard
Data ScientistHow to Target Coupon Users
Analytics & Experimentation
4
0
December 16, 2025
Coinbase
Medium
Data ScientistInvestigate Anomalies in Coinbase Wallet Engagement Metrics
Analytics & Experimentation
24
0
August 4, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.