Capital One
Hard
Data ScientistPresent and critique an airline delay analysis
Analytics & Experimentation
11
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Optiver
Hard
Software EngineerDesign and backtest a trading strategy
Analytics & Experimentation
4
0
August 13, 2025
Amazon
Medium
Data ScientistDiagnose Causes and Test Hypotheses for Metric Drop
Analytics & Experimentation
35
0
July 12, 2025
Uber
Hard
Data ScientistDesign station experiment with interference and rush-hour spillovers
Analytics & Experimentation
9
0
October 13, 2025
Figma
Hard
Software EngineerDefine and measure article trending
Analytics & Experimentation
10
0
September 6, 2025
Uber
Hard
Data ScientistEvaluate Push Notification Impact on Rideshare Supply Shortages
Analytics & Experimentation
17
0
August 4, 2025
Netflix
Medium
Data ScientistHow to Design Effective A/B Tests for Onboarding
Analytics & Experimentation
22
0
July 12, 2025
Uber
Hard
Data ScientistDesign an A/B test for promo-targeting models
Analytics & Experimentation
6
0
October 13, 2025
Airbnb
Medium
Data ScientistDefine Success Metrics and Experiment Plan for Product Development
Analytics & Experimentation
24
0
July 12, 2025
Shopify
Medium
Data ScientistAnalyze Causes of November and June Shopify Traffic Spikes
Analytics & Experimentation
6
0
August 4, 2025
LinkedIn
Hard
Data ScientistAssess LinkedIn Newsfeed Health
Analytics & Experimentation
27
0
July 12, 2025
DoorDash
Hard
Data ScientistDesign experiments for marketplace product changes
Analytics & Experimentation
4
0
February 9, 2026
Meta
Easy
Data ScientistDesign and evaluate a new group call feature
Analytics & Experimentation
8
0
December 8, 2025
Uber
Hard
Data ScientistDefine ride success metric for Uber
Analytics & Experimentation
8
0
October 13, 2025
LinkedIn
Medium
Data ScientistImprove Profile Completion Rate
Analytics & Experimentation
26
0
July 12, 2025
Meta
Hard
Data ScientistDetermine Success Metrics for New Group Video-Call Feature
Analytics & Experimentation
30
0
July 12, 2025
Gusto
Easy
Data ScientistCompute p-values for 2 variants vs control
Analytics & Experimentation
6
0
November 2, 2025
DoorDash
Hard
Data ScientistDiagnose cold-food spike and design experiments
Analytics & Experimentation
8
0
October 13, 2025
DoorDash
Hard
Data ScientistEvaluate and test a Top Dasher program
Analytics & Experimentation
9
0
October 13, 2025
Meta
Easy
Data ScientistEvaluate AI-assisted ads creation feature
Analytics & Experimentation
7
0
February 15, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.