TikTok
Medium
Data ScientistEvaluate Cohort Posting Patterns Using Metrics and Tests
Analytics & Experimentation
0
0
August 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Coinbase
Medium
Data ScientistDiagnose Retail Revenue Drop and Predict Ad Impact
Analytics & Experimentation
0
0
August 4, 2025
Meta
Medium
Data ScientistDiagnose spend drops, bots, and Stories
Analytics & Experimentation
0
0
January 25, 2026
Meta
Medium
Data ScientistHow would you evaluate Pixel issue alerts?
Analytics & Experimentation
0
0
January 20, 2026
Meta
Hard
Data ScientistAssess Need for Group Calls
Analytics & Experimentation
0
0
January 2, 2026
Airwallex
Medium
Data ScientistHow would you measure billboard impact?
Analytics & Experimentation
0
0
October 24, 2025
Capital One
Hard
Data ScientistDesign an experiment for delay drivers
Analytics & Experimentation
0
0
October 13, 2025
Amazon
Medium
Data ScientistIdentify Causes and Validate Web Product Performance Drop
Analytics & Experimentation
0
0
August 4, 2025
None
Easy
Data ScientistHow would you measure and experiment on harmful content?
Analytics & Experimentation
0
0
February 17, 2026
Hard
Data ScientistDefine goals and success metrics for subscriber-only features
Analytics & Experimentation
0
0
October 13, 2025
CloudTrucks
Medium
Data ScientistDetermine North-Star Metric for CloudTrucks Driver Platform
Analytics & Experimentation
0
0
August 4, 2025
Nextdoor
Medium
Data ScientistMeasure Local News Launch Success
Analytics & Experimentation
0
0
March 23, 2026
DoorDash
Medium
Data ScientistInvestigate LA Order Drop
Analytics & Experimentation
0
0
January 18, 2026
DoorDash
Medium
Data ScientistShould DoorDash add bicycle dashers?
Analytics & Experimentation
0
0
January 12, 2026
DoorDash
Hard
Data ScientistInvestigate Declining Successful Orders
Analytics & Experimentation
0
0
October 5, 2025
Airwallex
Medium
Data ScientistExplain Harassment Surge
Analytics & Experimentation
0
0
October 1, 2025
Meta
Hard
Data ScientistShould We Launch Group Calling?
Analytics & Experimentation
0
0
March 4, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.