OpenAI
Easy
Data ScientistEvaluate a free-trial A/B test
Analytics & Experimentation
3
0
October 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
DoorDash
Hard
Data ScientistDetermine Optimal Dasher Compensation Model and Diagnose Metric Drops
Analytics & Experimentation
5
0
August 4, 2025
Roblox
Hard
Data ScientistDetermine Player Preference for Local Game Creators
Analytics & Experimentation
27
0
July 12, 2025
Capital One
Medium
Data ScientistDesign A/B Test for Marketing Campaign Impact Evaluation
Analytics & Experimentation
31
0
August 4, 2025
Meta
Medium
Data ScientistTrack Metrics to Measure Push Notification Quality
Analytics & Experimentation
22
0
August 4, 2025
Coinbase
Medium
Data ScientistAnalyze Factors Behind 20% Retail Revenue Decline
Analytics & Experimentation
31
0
July 12, 2025
Meta
Hard
Data ScientistDesign and validate an ads feed experiment
Analytics & Experimentation
7
0
October 13, 2025
DoorDash
Hard
Data ScientistDesign an experiment for order batching
Analytics & Experimentation
5
0
October 13, 2025
DoorDash
Medium
Data ScientistDefine and Measure Merchant Variety's Impact on Consumers
Analytics & Experimentation
3
0
August 4, 2025
Intuit
Easy
Data ScientistDesign an experiment for pricing page redesign
Analytics & Experimentation
7
0
November 8, 2025
Intuit
Easy
Data ScientistDiagnose KPI anomaly and evaluate promotion/A-B test
Analytics & Experimentation
5
0
August 1, 2025
LinkedIn
Medium
Data ScientistResolve Conflicting A/B Test Results in Cities
Analytics & Experimentation
21
0
July 12, 2025
LinkedIn
Medium
Data ScientistIdentify Sales Professionals
Analytics & Experimentation
16
0
July 12, 2025
Meta
Medium
Data ScientistShould WhatsApp Launch Group Calls?
Analytics & Experimentation
7
0
March 14, 2026
Shopify
Hard
Data ScientistMeasure Shopify App Store Launch Success Effectively
Analytics & Experimentation
7
0
August 4, 2025
Meta
Medium
Data ScientistAssessing whether a new metric A is meaningful for News Feed
Analytics & Experimentation
26
0
July 12, 2025
Uber
Medium
Data ScientistHow to evaluate lowering ETA?
Analytics & Experimentation
8
0
February 1, 2026
DoorDash
Medium
Data ScientistDiagnose completed orders drop in Los Angeles
Analytics & Experimentation
6
0
January 25, 2026
DoorDash
Hard
Data ScientistDesign and analyze batching algorithm experiment
Analytics & Experimentation
3
0
October 13, 2025
Airbnb
Medium
Data ScientistDefine product success metrics
Analytics & Experimentation
6
0
September 6, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.