Instacart
Easy
Data ScientistHow would you investigate a metric decline?
Analytics & Experimentation
6
0
February 6, 2026
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Pinterest
Medium
Data ScientistDesign and assess video-pin increase experiment
Analytics & Experimentation
3
0
October 13, 2025
Airwallex
Easy
Data ScientistDiagnose conversion-rate time series and CTA swap
Analytics & Experimentation
4
0
October 1, 2025
Amazon
Medium
Data ScientistIdentify Issues and Redesign Customer-Conversion Chart
Analytics & Experimentation
17
0
August 4, 2025
LinkedIn
Medium
Data ScientistAnalyze Trends to Diagnose Decline in Job Applications
Analytics & Experimentation
17
0
July 12, 2025
Uber
Medium
Data ScientistEvaluate marketplace interventions
Analytics & Experimentation
6
0
March 22, 2026
Google
Medium
Data ScientistDesign pricing and multivariate button experiments
Analytics & Experimentation
4
0
October 13, 2025
Upstart
Hard
Data ScientistDecide to ship a signup experiment
Analytics & Experimentation
4
0
October 13, 2025
Pinterest
Medium
Data ScientistInterpret A/B results for video-pin increase
Analytics & Experimentation
6
0
October 13, 2025
Meta
Medium
Data EngineerDefine success metrics for a social feed
Analytics & Experimentation
5
0
September 6, 2025
DoorDash
Medium
Data ScientistDiagnose Decline in Delivery Success: Data, Hypotheses, Tests
Analytics & Experimentation
5
0
August 4, 2025
Google
Medium
Data ScientistEvaluate Auto-Reply Feature Success with Metrics and Experiments
Analytics & Experimentation
15
0
July 12, 2025
Meta
Medium
Data ScientistAnalyze Revenue Shifts to Identify Cannibalization Effects
Analytics & Experimentation
15
0
July 12, 2025
Google
Hard
Data ScientistDefine and apply Gmail user segments
Analytics & Experimentation
3
0
October 13, 2025
OneMain Financial
Medium
Data ScientistOptimize Credit-Card Strategy: Pricing, Limits, and Target Segments
Analytics & Experimentation
13
0
July 12, 2025
OneMain Financial
Medium
Data ScientistMaximize credit card portfolio profit
Analytics & Experimentation
4
0
October 13, 2025
Uber
Hard
Data ScientistMeasure driver experience quantitatively
Analytics & Experimentation
3
0
October 13, 2025
Meta
Medium
Data ScientistEvaluate AI-assisted ad creation
Analytics & Experimentation
3
0
March 1, 2026
Google
Medium
Data ScientistDesign an A/B test with guardrails and SRM checks
Analytics & Experimentation
3
0
October 13, 2025
DoorDash
Medium
Data ScientistDiagnose Delivery Delays: Key Metrics and Experiment Design
Analytics & Experimentation
5
0
August 4, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.