OpenAI
Medium
Data ScientistHow would you evaluate a free-trial A/B test?
Analytics & Experimentation
4
0
October 8, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Apple
Medium
Data ScientistInvestigate Conversion Drop: Metrics, Analyses, Techniques Explained
Analytics & Experimentation
22
0
July 12, 2025
LinkedIn
Hard
Data ScientistLinkedIn Product Case Opportunity Sizing
Analytics & Experimentation
10
0
April 30, 2025
OpenAI
Hard
Data ScientistAssess free-month promotion impact
Analytics & Experimentation
5
0
January 22, 2026
Meta
Easy
Data ScientistHow would you compare Facebook vs Instagram Stories?
Analytics & Experimentation
8
0
November 1, 2025
Google
Hard
Data ScientistExperimentally evaluate jogging-route recommendations
Analytics & Experimentation
6
0
October 13, 2025
Meta
Easy
Data ScientistHow would you evaluate pixel-issue notifications?
Analytics & Experimentation
7
0
February 18, 2026
DoorDash
Medium
Data ScientistMeasure Impact of Merchant Variety on Consumer Experience
Analytics & Experimentation
17
0
August 4, 2025
Instacart
Easy
Data ScientistDesign a pricing experiment with network effects
Analytics & Experimentation
9
0
February 6, 2026
Uber
Hard
Data ScientistImprove Estimated Time of Arrival for Uber Riders
Analytics & Experimentation
5
0
August 4, 2025
Meta
Medium
Data ScientistDesign "Restaurants You May Know" Recommendation Algorithm
Analytics & Experimentation
33
0
July 12, 2025
Uber
Hard
Data ScientistDesign an RCT for app-open discount
Analytics & Experimentation
5
0
October 13, 2025
Uber
Medium
Data ScientistEvaluate Auto-Play Impact with Key Metrics and Experiment Design
Analytics & Experimentation
25
0
July 12, 2025
Meta
Easy
Data ScientistAnalyze and mitigate fake advertiser accounts
Analytics & Experimentation
6
0
February 15, 2026
Netflix
Hard
Data ScientistPlan and analyze a ranking A/B test
Analytics & Experimentation
7
0
October 13, 2025
Airbnb
Hard
Data ScientistEstimate Causal Impact Using Synthetic Control Methods
Analytics & Experimentation
13
0
July 12, 2025
DoorDash
Medium
Data ScientistBoost App Installs: Analyze and Experiment with Conversion Funnel
Analytics & Experimentation
21
0
July 12, 2025
Meta
Medium
Data ScientistIdentify Potential Users for Instagram Shopping Tab Adoption
Analytics & Experimentation
31
0
July 12, 2025
Meta
Easy
Data ScientistCompare performance of FB vs IG Stories
Analytics & Experimentation
7
0
February 16, 2026
Instacart
Hard
Data ScientistCompute duration and stopping rules correctly
Analytics & Experimentation
6
0
October 13, 2025
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.