Analyze DoorDash marketplace product decisions

You are a product-focused data scientist at DoorDash. Discuss how you would approach the following three product analytics and experimentation problems.

1. Top Dasher program DoorDash is considering changes to the Top Dasher program, which gives certain dashers additional benefits and may affect fulfillment quality, dasher incentives, and marketplace balance.
   • What are the main pros and cons of the program from the perspectives of consumers, dashers, merchants, and DoorDash?
   • What success metrics, secondary metrics, and guardrail metrics would you define?
   • What should the randomization unit be for an experiment, and why?
   • If the experiment’s primary metric is lower in treatment than in control, how would you investigate before deciding whether to ship, iterate, or roll back?
2. Order cancellation rate is increasing Suppose the overall order cancellation rate has risen materially over the last several weeks.
   • How would you diagnose the problem?
   • Which parts of the organization or product funnel could be contributing to the increase?
   • How would you identify likely root causes rather than just correlations?
   • How would you test your hypotheses and prioritize actions?
3. Merchant-created promotions vs. automatically generated promotions DoorDash is deciding between two promotion systems for merchants:
   • merchants manually create and configure promotions themselves, or
   • DoorDash automatically recommends or launches promotions on their behalf.
   Compare the pros and cons of the two approaches, including trade-offs in merchant control, adoption, incremental demand, profitability, and marketplace health. Then design an experiment to evaluate the better approach:
   • define the key product and business metrics,
   • choose the right randomization unit,
   • discuss spillover effects and selection bias,
   • and explain how you would interpret the results if different stakeholders benefit in different ways.