Design A/B Test to Evaluate Algorithm's Revenue Impact
Company: DoorDash
Role: Data Scientist
Category: Statistics & Math
Difficulty: hard
Interview Round: Technical Screen
##### Scenario
You are responsible for evaluating the lift of a new recommendation algorithm via online experiments.
##### Question
Design an A/B test to measure the algorithm’s impact on revenue: define hypotheses, choose unit of randomization, compute required sample size, and detail success metrics. After the experiment you obtain p = 0.08 for revenue lift; interpret this result and recommend whether to ship. Explain how you would estimate causal impact if randomization were not possible; compare methods such as difference-in-differences, propensity score matching, and instrumental variables.
##### Hints
Demonstrate knowledge of hypothesis testing, power analysis, Type I/II errors, and causal inference techniques.
Quick Answer: DoorDash statistics and experimentation prompt on A/B testing a recommendation algorithm's revenue impact, covering hypotheses, randomization, sample size, heavy-tailed revenue, p=0.08 interpretation, and causal alternatives.