TikTok
Medium
Data ScientistDiagnose Search Issues with Relevant Metrics and Solutions
Analytics & Experimentation
15
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Google
Medium
Data ScientistSegment Gmail Users for Targeted Marketing and Product Enhancement
Analytics & Experimentation
11
0
July 12, 2025
Meta
Medium
Data Scientist[Analytics Reasoning] Impact of Malicious Accounts on Meta
Analytics & Experimentation
9
0
April 7, 2025
DoorDash
Hard
Data ScientistDiagnose LA completed-order drop and design experiment
Analytics & Experimentation
5
0
October 13, 2025
Pinterest
Medium
Data ScientistDesign and interpret video-pins experiment results
Analytics & Experimentation
3
0
October 13, 2025
TikTok
Medium
Data ScientistAnalyze promo anomaly and design risk guardrails
Analytics & Experimentation
5
0
October 13, 2025
Instacart
Hard
Data ScientistUse regression vs cohorts for A/B estimation
Analytics & Experimentation
2
0
October 13, 2025
DoorDash
Hard
Data ScientistDiagnose and reduce cold-food refund costs
Analytics & Experimentation
3
0
October 13, 2025
Meta
Hard
Data ScientistEvaluate Success Metrics for Facebook Groups and New Features
Analytics & Experimentation
11
0
July 12, 2025
Instacart
Hard
Data ScientistDiagnose Sunday Miami same‑day outages
Analytics & Experimentation
3
0
October 13, 2025
LinkedIn
Medium
Data ScientistDecide best email variant using stratified A/B analysis
Analytics & Experimentation
3
0
October 13, 2025
Pinterest
Medium
Data ScientistHow would you evaluate a carousel launch?
Analytics & Experimentation
5
0
March 10, 2026
Reddit
Easy
Data ScientistDesign A/B test for AI chat box
Analytics & Experimentation
3
0
December 11, 2025
Instacart
Medium
Data ScientistEvaluate Miami Ultrafast impact on orders
Analytics & Experimentation
3
0
October 13, 2025
Uber
Hard
Data ScientistDesign promo experiment and explain correlation
Analytics & Experimentation
4
0
September 16, 2025
PayPal
Medium
Data ScientistBoost User Login Rate: Key Metrics to Monitor
Analytics & Experimentation
2
0
August 4, 2025
Roku
Medium
Data ScientistAnalyze Profit Decline: Data Collection and Hypothesis Testing
Analytics & Experimentation
10
0
July 12, 2025
TikTok
Medium
Data ScientistSQL Queries and Analysis on Bad Advertisers
Analytics & Experimentation
10
0
June 29, 2025
Meta
Easy
Data ScientistHow would you evaluate upranking shop ads?
Analytics & Experimentation
4
0
February 12, 2026
LinkedIn
Easy
Data ScientistDo US members upload more videos than non-US?
Analytics & Experimentation
3
0
February 1, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.