Google
Medium
Data ScientistDesign A/B Test to Isolate Product Usage Drop Causes
Analytics & Experimentation
57
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Meta
Easy
Data ScientistHow would you define and use retention metrics?
Analytics & Experimentation
9
0
February 18, 2026
Meta
Easy
Data ScientistDesign measurement to detect fake accounts
Analytics & Experimentation
7
1
November 16, 2025
Google
Medium
Data ScientistMeasure causal impact of YouTube ads
Analytics & Experimentation
13
0
October 13, 2025
PayPal
Medium
Data ScientistAnalyze Transactions for Risk and Implement Mitigation Strategies
Analytics & Experimentation
56
0
August 4, 2025
Atlassian
Medium
Data ScientistDiagnose Checkout Rate Drop: Steps and Analyses
Analytics & Experimentation
51
0
August 4, 2025
Meta
Hard
Data ScientistEvaluate the Health of Facebook Groups
Analytics & Experimentation
49
0
July 12, 2025
Instacart
Medium
Data ScientistMeasure Ultrafast Delivery's Impact Using Synthetic Control Method
Analytics & Experimentation
54
0
July 12, 2025
DoorDash
Hard
Data ScientistEvaluate a new ranking model
Analytics & Experimentation
8
0
February 6, 2026
Meta
Medium
Data ScientistDiagnosing a drop in total ads revenue
Analytics & Experimentation
31
1
July 12, 2025
Airbnb
Hard
Data ScientistEstimate impact of global launch without holdout
Analytics & Experimentation
12
0
October 13, 2025
Etsy
Medium
Data ScientistMeasure and Improve Listing Quality with Key Metrics
Analytics & Experimentation
57
0
August 4, 2025
Shopify
Easy
Data ScientistMeasure App Store success and debug funnel anomaly
Analytics & Experimentation
9
0
November 21, 2025
Roblox
Medium
Data ScientistEvaluate Impact of New Roblox Homepage Tab
Analytics & Experimentation
50
0
July 12, 2025
Meta
Medium
Data ScientistQuality and frequency control for push notifications
Analytics & Experimentation
54
0
July 12, 2025
Apple
Medium
Data ScientistDesign A/B Test for Search Feature Effectiveness
Analytics & Experimentation
33
0
August 4, 2025
DoorDash
Hard
Data ScientistDecompose and optimize delivery operational costs
Analytics & Experimentation
9
0
October 13, 2025
DoorDash
Hard
Data ScientistExperiment on increasing order notifications
Analytics & Experimentation
11
0
October 13, 2025
Uber
Hard
Data ScientistDesign an ETA experiment under interference
Analytics & Experimentation
11
0
October 13, 2025
DoorDash
Easy
Data ScientistDesign experiments for payments, search, and promotions
Analytics & Experimentation
11
0
February 5, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.