Capital One
Medium
Data ScientistDecide Which Show to Renew
Analytics & Experimentation
1
0
February 11, 2026
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
LinkedIn
Hard
Data ScientistHow to diagnose traffic and measure relevance?
Analytics & Experimentation
0
0
January 21, 2026
Meta
Medium
Data ScientistHow to decide if users need a new feature
Analytics & Experimentation
1
0
November 2, 2025
Intuit
Easy
Product AnalystDesign an experiment to evaluate an onboarding progress bar
Analytics & Experimentation
1
0
October 21, 2025
Meta
Hard
Data ScientistDesign and analyze end-to-end A/B test
Analytics & Experimentation
1
0
October 13, 2025
Meta
Medium
Data ScientistDesign pre-launch plan and cluster A/B test
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistProve source growth is cannibalization, not incremental
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistDesign a small-sample launch experiment in Europe
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistEvaluate shopping tab pre- and post-launch
Analytics & Experimentation
1
0
October 13, 2025
TikTok
Hard
Data ScientistDesign robust metrics for a feature launch
Analytics & Experimentation
0
0
October 13, 2025
TikTok
Hard
Data ScientistDesign causal measurement without randomization
Analytics & Experimentation
1
0
October 13, 2025
Meta
Medium
Data ScientistEvaluate emoji reactions launch
Analytics & Experimentation
1
0
October 13, 2025
PayPal
Easy
Data ScientistDiagnose drop in shopper order acceptance
Analytics & Experimentation
1
0
October 10, 2025
Spokeo
Medium
Data ScientistCalculate Customer Lifetime Value for Spokeo Using Models
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistDetermine User Need for In-App Video Call Feature
Analytics & Experimentation
1
0
August 4, 2025
Amazon
Medium
Data ScientistDetermine Discount's Effect on Conversion Rate with A/B Testing
Analytics & Experimentation
1
0
August 4, 2025
TikTok
Medium
Data ScientistAnalyze Trade-off Between DAU Growth and Ad Revenue
Analytics & Experimentation
0
0
August 4, 2025
Capital One
Medium
Data ScientistEstimate Venmo Revenue and Boost User Engagement Metrics
Analytics & Experimentation
1
0
August 4, 2025
Meta
Hard
Data ScientistMeasure whether posts strengthen friendships
Analytics & Experimentation
1
0
July 28, 2025
Amazon
Medium
Software EngineerBrainstorm a business problem approach
Analytics & Experimentation
1
0
July 15, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.