Pinterest
Hard
Data ScientistInvestigate Homepage Experiment Without Control Group: Methods and Metrics
Analytics & Experimentation
94
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Google
Hard
Data ScientistDiagnose Google Meet Disconnections and Assess Business Impact
Analytics & Experimentation
91
0
July 12, 2025
DoorDash
Hard
Data ScientistDesign Experiments to Evaluate Courier Initiatives Effectively
Analytics & Experimentation
83
0
July 12, 2025
Netflix
Medium
Data ScientistEstimate ATE of personalization on streaming
Analytics & Experimentation
15
0
March 5, 2026
Capital One
Medium
Data ScientistCalculate Profitability and Evaluate Partnership for Credit Card Portfolio
Analytics & Experimentation
95
0
July 12, 2025
Robinhood
Medium
Data ScientistDiagnose Decline in First Day Funding Rate
Analytics & Experimentation
85
0
August 4, 2025
Meta
Hard
Data ScientistDefine Success Metrics for Circle Feature Evaluation
Analytics & Experimentation
80
0
August 4, 2025
Roblox
Easy
Data ScientistHow to estimate feature impact on usage time
Analytics & Experimentation
15
0
January 17, 2026
Meta
Medium
Data ScientistEvaluate Metrics for Restaurant-Feature Impact and Engagement Trade-offs
Analytics & Experimentation
85
0
July 12, 2025
Google
Hard
Data ScientistDiagnose YouTube Usage Decline: Key Metrics and Segmentation
Analytics & Experimentation
76
0
July 12, 2025
Uber
Easy
Data ScientistDesign metrics and A/B test for maps and ETA
Analytics & Experimentation
16
0
November 9, 2025
Meta
Hard
Data ScientistEvaluate Messenger's P2P Payments Feature for Business Viability
Analytics & Experimentation
84
0
July 12, 2025
Lyft
Medium
Data ScientistInvestigate Sudden Metric Changes and Design A/B Test
Analytics & Experimentation
64
0
August 4, 2025
Capital One
Medium
Data ScientistExplain App Growth Strategy and Key Performance Metrics
Analytics & Experimentation
93
0
July 12, 2025
DoorDash
Hard
Data ScientistEvaluate Top-Dasher Program's Benefits and Challenges
Analytics & Experimentation
67
0
August 4, 2025
Meta
Easy
Data ScientistHow to measure harmful-content severity and run experiments
Analytics & Experimentation
23
0
February 18, 2026
Uber
Medium
Data ScientistDetermine Sample Size for Promotion Campaign A/B Test
Analytics & Experimentation
70
0
August 4, 2025
Meta
Medium
Data ScientistEvaluating the Facebook ‘Memory’ feature
Analytics & Experimentation
93
0
July 12, 2025
Uber
Medium
Data ScientistEvaluate Rider-Incentive Program Impact with Key Metrics
Analytics & Experimentation
73
0
July 12, 2025
Meta
Medium
Data ScientistDefine Metrics and Account for Network and Novelty Effects
Analytics & Experimentation
63
0
July 12, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.