LinkedIn
Medium
Data ScientistEvaluate an email test with confounding
Analytics & Experimentation
1
0
July 8, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Wayfair
Easy
Data ScientistDerive insights and improve complaint resolutions
Analytics & Experimentation
1
0
February 18, 2026
Meta
Medium
Data ScientistAssess ranking change and design experiment
Analytics & Experimentation
0
0
January 21, 2026
Meta
Medium
Data ScientistEvaluate new-product notification feature
Analytics & Experimentation
1
0
January 5, 2026
Etsy
Hard
Data ScientistRun a clean A/B test for autocomplete
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistDiagnose rising account switching and falling actives
Analytics & Experimentation
1
0
October 13, 2025
Meta
Hard
Data ScientistValidate needs and benchmark competitor adoption
Analytics & Experimentation
1
0
October 13, 2025
Capital One
Medium
Data ScientistSegment 500k users into three groups
Analytics & Experimentation
0
0
October 13, 2025
Meta
Hard
Data ScientistMeasure notification impact and set guardrails
Analytics & Experimentation
1
0
October 13, 2025
Other
Medium
Data ScientistFind and fix metric drops systematically
Analytics & Experimentation
0
0
October 13, 2025
Other
Hard
Data ScientistTranslate goals into robust product metrics
Analytics & Experimentation
1
0
October 13, 2025
Capital One
Medium
Data ScientistDecide whether to sell both SKUs
Analytics & Experimentation
0
0
October 13, 2025
NVIDIA
Medium
Software EngineerDefine developer-centric usability metrics
Analytics & Experimentation
1
0
September 6, 2025
Chime
Medium
Data ScientistDesign an Effective A/B Test for Algorithm Launch
Analytics & Experimentation
1
0
August 4, 2025
Meta
Medium
Data ScientistDesign A/B Test to Evaluate New Video-Feed Feature
Analytics & Experimentation
1
0
August 4, 2025
Capital One
Medium
Data ScientistEvaluate Profit Margins and Future Trends for Vegan Burgers
Analytics & Experimentation
1
0
August 4, 2025
TikTok
Medium
Data ScientistDiagnose Decline in User Engagement and Experience Quality
Analytics & Experimentation
0
0
August 4, 2025
Meta
Medium
Data ScientistHow to evaluate new listing notifications?
Analytics & Experimentation
1
0
March 2, 2026
Capital One
Medium
Data ScientistHow would you choose between shows?
Analytics & Experimentation
1
0
January 9, 2026
PayPal
Medium
Data ScientistAnalyze an A/B test and present recommendation
Analytics & Experimentation
0
0
December 10, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.