Coinbase
Medium
Data ScientistInvestigate Anomalies in Coinbase Wallet Engagement Metrics
Analytics & Experimentation
24
0
August 4, 2025
A/B Testing Interview Questions
A/B testing questions are central to data science and product analytics interviews at companies like Meta, Google, Netflix, and Airbnb.
Expect questions on experiment design, randomization units, sample size calculation, multiple comparisons, and metric selection.
Interviewers evaluate your statistical rigor, practical judgment, and ability to communicate experiment results.
837Questions
79Companies
Easy 99 • Medium 59 • Hard 12Difficulty mix
Common A/B testing interview patterns
- Designing an experiment for a product change
- Calculating sample size and experiment duration
- Choosing between one-sided and two-sided tests
- Handling multiple comparisons and peeking
- Interpreting results with novelty or primacy effects
- Network effects and interference between test groups
A/B testing interview questions
Filter and sort
Capital One
Hard
Product AnalystEvaluate food-court profitability and membership strategy
Analytics & Experimentation
4
0
April 21, 2025
Capital One
Easy
Data ScientistDiagnose Flight Delays and Burger Launch
Analytics & Experimentation
4
0
February 28, 2026
Meta
Medium
Data ScientistDesign analysis to test social vs game engagement
Analytics & Experimentation
4
1
October 13, 2025
Shopify
Medium
Data ScientistDesign robust experiment for ambiguous core change
Analytics & Experimentation
6
0
October 13, 2025
Uber
Hard
Data ScientistEvaluate impact without randomized experiments
Analytics & Experimentation
4
0
October 13, 2025
Netflix
Hard
Data ScientistDesign A/B Test for Streaming Feature Network Effects
Analytics & Experimentation
5
0
August 4, 2025
LinkedIn
Hard
Data ScientistEvaluate 'Job You May Be Interested In' Recommender
Analytics & Experimentation
17
0
July 12, 2025
Roblox
Hard
Data ScientistDesign ad revenue A/B with guardrails
Analytics & Experimentation
4
0
October 13, 2025
Meta
Medium
Data EngineerDefine and analyze product metrics
Analytics & Experimentation
6
1
September 6, 2025
LinkedIn
Medium
Data ScientistTest If Initial Video Uploads Are Shorter Than Later Ones
Analytics & Experimentation
24
0
July 12, 2025
LinkedIn
Medium
Data ScientistSingle Queue vs Multiple Queues — Service Design
Analytics & Experimentation
14
0
July 12, 2025
DoorDash
Easy
Product AnalystInvestigate LA successful orders drop
Analytics & Experimentation
5
0
February 19, 2026
Uber
Hard
Data ScientistDesign a robust email A/B test
Analytics & Experimentation
6
0
October 13, 2025
Meta
Hard
Data ScientistImplement Clustered Sampling to Mitigate Network Effects in Testing
Analytics & Experimentation
21
0
July 12, 2025
Meta
Hard
Data ScientistEvaluate Recommendation Feature with Historical Data Analysis
Analytics & Experimentation
20
0
July 12, 2025
LinkedIn
Hard
Data ScientistOne of the most comprehensive LinkedIn DS Product Cases!
Analytics & Experimentation
11
0
April 30, 2025
Meta
Medium
Data ScientistAssess Demand for Group Video Chat
Analytics & Experimentation
4
0
March 14, 2026
Bytedance
Hard
Data AnalystDefine Ultra success and detect suspicious transactions
Analytics & Experimentation
5
0
February 23, 2026
Capital One
Easy
Data ScientistHow would you decide to cancel a TV show?
Analytics & Experimentation
3
0
February 12, 2026
Common mistakes in A/B testing interviews
- Not specifying the randomization unit (user vs session vs page)
- Peeking at results before reaching the required sample size
- Ignoring practical significance when statistical significance is achieved
- Not considering guardrail metrics
- Failing to account for novelty effects in short experiments
How A/B testing questions are evaluated
Structure your experiment design: hypothesis, metrics, unit, sample size, duration.
Discuss what could go wrong and how you would detect it.
Show ability to make a recommendation even when results are ambiguous.
Related analytics concepts
A/B Testing Interview FAQs
How do you determine the sample size for an A/B test?
Use a power analysis with inputs: baseline metric, minimum detectable effect (MDE), significance level (alpha, usually 0.05), and power (usually 0.80). Larger effects need fewer samples. For small MDE on rare events, you may need millions of users.
What is the difference between statistical and practical significance?
Statistical significance means the observed difference is unlikely due to chance (p-value < alpha). Practical significance means the effect is large enough to matter for the business. A statistically significant 0.01% lift may not be worth the engineering cost to ship.