Meta
Medium
Data ScientistDetermine Metrics for Group-Video Calling Experiment Success
Analytics & Experimentation
81
0
July 12, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
DoorDash
Hard
Data ScientistEvaluate Account-Partner Onboarding with Success Metrics
Analytics & Experimentation
63
0
August 4, 2025
CVS Health
Medium
Data ScientistDesign Experiments for Causal Inference in Marketing Analytics
Analytics & Experimentation
73
0
August 4, 2025
Uber
Medium
Data ScientistEvaluate Promotion Campaign Effectiveness with A/B Testing
Analytics & Experimentation
65
0
July 12, 2025
Meta
Medium
Data ScientistMeasuring and mitigating fake news on Facebook
Analytics & Experimentation
73
0
July 12, 2025
Meta
Medium
Data ScientistDesign Experiment to Evaluate New Video-Ad Effectiveness
Analytics & Experimentation
79
0
July 12, 2025
DoorDash
Medium
Data ScientistDiagnose Cold Food Deliveries with Key Metrics Analysis
Analytics & Experimentation
31
0
July 12, 2025
Capital One
Hard
Data ScientistDecide on vegan-burger R&D investment
Analytics & Experimentation
16
0
October 13, 2025
Meta
Medium
Data ScientistEvaluating Instagram’s one‑tap account switcher
Analytics & Experimentation
75
0
July 12, 2025
Meta
Hard
Data ScientistEvaluate Social Media's Brand Advertising Effectiveness
Analytics & Experimentation
73
0
July 12, 2025
Meta
Medium
Data ScientistEvaluate AI-assisted ad creation
Analytics & Experimentation
19
0
February 23, 2026
Waymo
Easy
Data ScientistHow validate a driving simulation is realistic?
Analytics & Experimentation
14
0
January 17, 2026
PayPal
Medium
Data ScientistMaster A/B Testing: Key Concepts and Methodologies Explained
Analytics & Experimentation
76
0
July 12, 2025
Anthropic
Hard
Software EngineerHow do you design an A/B experiment?
Analytics & Experimentation
12
0
March 1, 2026
Meta
Hard
Data ScientistBuilding a restaurant‑recommendation feature with Nearby Friends signals
Analytics & Experimentation
69
0
July 12, 2025
DoorDash
Hard
Data ScientistDesign analytics for a new-market launch
Analytics & Experimentation
18
0
October 13, 2025
Meta
Hard
Data ScientistEvaluate Chatbot Launch: Value, Risks, Impact, Success Metrics
Analytics & Experimentation
53
0
July 12, 2025
Google
Medium
Data ScientistDesign A/B Test for Subscription Price Increase Effectiveness
Analytics & Experimentation
65
0
July 12, 2025
TikTok
Hard
Data ScientistMeasure Billboard Campaign Effectiveness and Engagement Quantification
Analytics & Experimentation
71
0
July 12, 2025
DoorDash
Medium
Data ScientistHow to test bike delivery?
Analytics & Experimentation
17
0
March 1, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.