Lyft
Hard
Data ScientistInvestigate Causes and Effects of Dynamic Pricing on ETAs
Analytics & Experimentation
82
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
DoorDash
Medium
Data ScientistInvestigate Causes of Cold Meal Deliveries
Analytics & Experimentation
37
0
July 12, 2025
Meta
Medium
Data ScientistEvaluate Instagram's Short-Video Recommender System Success
Analytics & Experimentation
107
0
July 12, 2025
LinkedIn
Medium
Data ScientistDiagnose Job-Application Decline: Funnel Stages and KPIs Analysis
Analytics & Experimentation
109
0
August 4, 2025
Meta
Hard
Data ScientistUncover User Needs for Group Calling Effectively
Analytics & Experimentation
109
0
July 12, 2025
Meta
Medium
Data ScientistEvaluate the Success of Instagram Checkout
Analytics & Experimentation
87
0
July 12, 2025
Upstart
Hard
Data ScientistExplain Treatment Results and Recommend Launch Criteria for Experiments
Analytics & Experimentation
102
0
August 4, 2025
Meta
Medium
Data ScientistHow to Validate Friends' Content Engagement Hypothesis?
Analytics & Experimentation
97
0
July 12, 2025
Uber
Hard
Data ScientistDesign an Uber A/B experiment end-to-end
Analytics & Experimentation
21
0
October 13, 2025
Capital One
Medium
Data ScientistEvaluate Financial Feasibility of Ride-Sharing Service
Analytics & Experimentation
106
0
July 12, 2025
DoorDash
Hard
Data ScientistAssess Success Criteria for Bike-Courier Delivery Launch
Analytics & Experimentation
71
0
July 12, 2025
Meta
Easy
Data ScientistEvaluate new shop-ads ranking algorithm
Analytics & Experimentation
28
0
January 17, 2026
Meta
Hard
Data ScientistAnalyze Free Shuttle Impact on Employee Participation Rates
Analytics & Experimentation
101
0
July 12, 2025
Uber
Hard
Data ScientistDesign a switchback and choose block length
Analytics & Experimentation
21
0
October 13, 2025
Uber
Hard
Data ScientistAnalyze T2 Results and Recommend Launch Strategy
Analytics & Experimentation
89
1
August 4, 2025
DoorDash
Hard
Data ScientistEvaluate Account-Partner Performance with Observational Data Analysis
Analytics & Experimentation
91
0
August 4, 2025
Uber
Medium
Data ScientistEvaluate a cold-start rating launch
Analytics & Experimentation
14
0
April 6, 2026
Capital One
Medium
Data ScientistCalculate Profit and Analyze Vegan Burger Market Trends
Analytics & Experimentation
98
0
August 4, 2025
Meta
Medium
Data ScientistAdvertising for local businesses boosting popular posts
Analytics & Experimentation
101
0
July 12, 2025
Amazon
Hard
Data ScientistDesign A/B Test for New Amazon Recommendation Module
Analytics & Experimentation
100
0
August 4, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.