Company plans to launch a promotion campaign (e.g., 20% off when spending $40) and wants to evaluate it with an A/B test.

Why might the business introduce this promotion campaign and which success metrics would you monitor? How would you determine the required sample size for the study?  What inputs and assumptions must be considered? Revenue will be the primary metric.  The PM sets the minimum detectable effect (MDE) to 0.5%.  How do you interpret this value and how will it influence sample size? If the observed lift is 0.3% and not statistically significant, how would you interpret the result and communicate it to the PM?

Think about business objectives, primary/secondary metrics, variance, baseline, power, Type-I/II errors, and explaining small, non-significant effects to stakeholders.

Uber Data Scientist interview question: Determine Sample Size for Promotion Campaign A/B Test. {"blocks": [{"key": "9a79f8c4", "text": "Scenario", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {...