Google
Medium
Data ScientistEvaluate Optimal Jogging Routes Feature with A/B Testing
Analytics & Experimentation
9
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
DoorDash
Hard
Data ScientistDesign DoorDash Marketplace Experiments
Analytics & Experimentation
5
0
February 10, 2026
Pinterest
Hard
Data ScientistAnalyze a geo rollout and interpret charts
Analytics & Experimentation
5
0
October 13, 2025
Uber
Hard
Data ScientistEvaluate New Model's Impact on Rider and Driver Experience
Analytics & Experimentation
5
0
August 4, 2025
DoorDash
Easy
Data ScientistImprove biker delivery with metrics and levers
Analytics & Experimentation
6
0
July 1, 2025
TikTok
Easy
Data ScientistDefine Ultra success metrics and detect suspicious transactions
Analytics & Experimentation
4
0
February 4, 2026
Google
Hard
Data ScientistDesign an Unbiased Upgrade Experiment
Analytics & Experimentation
6
0
December 29, 2025
Instacart
Hard
Data ScientistAnalyze A/B test with revenue–cost tradeoffs
Analytics & Experimentation
5
0
October 13, 2025
Robinhood
Easy
Data ScientistDesign an experiment to launch fractional shares
Analytics & Experimentation
4
0
October 1, 2025
Coinbase
Medium
Data ScientistDiagnose Revenue Decline: Key Analyses and Metrics
Analytics & Experimentation
18
0
August 4, 2025
DoorDash
Easy
Data ScientistDesign a Top Dasher experiment with interference
Analytics & Experimentation
5
0
July 1, 2025
DoorDash
Hard
Data ScientistHow would you test product changes?
Analytics & Experimentation
5
0
March 10, 2026
Google
Hard
Data ScientistDesign a Causal Upgrade Experiment
Analytics & Experimentation
6
0
December 3, 2025
Google
Medium
Data ScientistHow do you diagnose a ratio metric change
Analytics & Experimentation
5
0
November 24, 2025
Shopify
Easy
Data ScientistHow would you measure App Store launch success?
Analytics & Experimentation
4
0
November 8, 2025
Google
Medium
Data ScientistDiagnose 10–11% usage drop across geos
Analytics & Experimentation
3
0
October 13, 2025
TikTok
Hard
Data ScientistMeasure Ads Manager effectiveness end-to-end
Analytics & Experimentation
5
0
October 13, 2025
Robinhood
Medium
Data ScientistEvaluate Core Metrics for New Product Feature Launch
Analytics & Experimentation
5
0
August 4, 2025
OneMain Financial
Medium
Data ScientistCalculate Profit-Maximizing Price and Validate with Additional Data
Analytics & Experimentation
15
0
July 12, 2025
Coinbase
Medium
Data ScientistEstimate Successful Sign-ups from Super Bowl QR Code Ad
Analytics & Experimentation
19
0
July 12, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.