Amazon
Hard
Data ScientistDesign and analyze pricing-page A/B test
Analytics & Experimentation
1
0
October 13, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Flatiron Health
Medium
Data ScientistDesign success and guardrail metrics
Analytics & Experimentation
2
0
October 13, 2025
Thumbtack
Hard
Data ScientistDesign a robust pro-ranking A/B test
Analytics & Experimentation
2
0
October 13, 2025
Roblox
Hard
Data ScientistEvaluate friend-interaction feature with network interference
Analytics & Experimentation
2
0
October 13, 2025
PayPal
Easy
Data ScientistPresent an A/B test project review
Analytics & Experimentation
2
0
October 10, 2025
Roblox
Easy
Data ScientistCompute diff-in-diff and pretrend flag
Analytics & Experimentation
2
0
October 3, 2025
Coinbase
Easy
Data ScientistEvaluate campaign success and decide new trading pair
Analytics & Experimentation
3
0
September 20, 2025
Meta
Medium
Data ScientistDetermine High-Quality Notifications with CTR Analysis
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistDesign Metrics to Track and Analyze Spam Impact
Analytics & Experimentation
2
0
August 4, 2025
Meta
Medium
Data ScientistDetermine Facebook's Restaurant Recommendation Viability Using Data
Analytics & Experimentation
5
0
July 12, 2025
LinkedIn
Easy
Data ScientistResolve Simpson’s paradox in A/B email test
Analytics & Experimentation
1
0
February 11, 2026
Meta
Medium
Product AnalystHow would you drive product growth?
Analytics & Experimentation
1
0
January 16, 2026
Lyft
Hard
Data ScientistHow to target commute coupon users?
Analytics & Experimentation
2
0
December 12, 2025
Gusto
Medium
Data ScientistHow do you test two variants vs control?
Analytics & Experimentation
2
0
November 30, 2025
TikTok
Easy
Data ScientistPlan DS approach for biker delivery project
Analytics & Experimentation
1
0
November 27, 2025
Amazon
Easy
Data ScientistHow would you analyze and test a price increase?
Analytics & Experimentation
1
0
November 20, 2025
PayPal
Easy
Data ScientistReduce airport cancellations under causal constraints
Analytics & Experimentation
1
0
November 3, 2025
Meta
Hard
Data ScientistSize opportunity and prioritize experiments
Analytics & Experimentation
2
0
October 13, 2025
Snowflake
Hard
Data ScientistDesign an A/B test for ML model launch
Analytics & Experimentation
2
0
October 13, 2025
Amazon
Hard
Data ScientistEvaluate concession gift-card policy with DID
Analytics & Experimentation
1
0
October 13, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.