Diagnose completed orders drop in Los Angeles

You are a data scientist at DoorDash supporting the consumer pricing team. The number of completed delivery orders in Los Angeles has dropped meaningfully over the last two weeks relative to both historical baseline and similar large cities.

How would you investigate the issue end-to-end?

In your answer, cover:

1. Problem framing and validation
   • How would you confirm the drop is real rather than seasonality, reporting latency, instrumentation bugs, or a temporary outage?
   • What benchmarks would you use: week-over-week, year-over-year, pre/post launch, and comparison to matched control cities?
2. Metrics and funnel decomposition Be explicit about the metrics you would examine and how they relate to completed orders. For example:
   • Demand : app opens, sessions, store views, checkout starts, order attempts
   • Pricing and conversion : basket size, menu price index, delivery fee, service fee, surge or small-order fees, promotions, DashPass/member mix, checkout conversion
   • Marketplace health : merchant availability, out-of-stock rate, merchant acceptance rate, courier supply, assignment time, ETA, cancellations
   • Outcome metrics : completed orders, completed orders per active consumer, gross order value, contribution margin
3. Hypotheses Generate and prioritize plausible explanations, including:
   • a pricing change that reduced conversion
   • lower courier supply or higher ETAs
   • merchant outages or lower assortment availability
   • app/checkout product regressions
   • external factors such as weather, major events, regulation, or competitor promotions
   • mix shift across neighborhoods, user cohorts, or dayparts
4. Segmentation and causal reasoning
   • How would you segment the analysis: new vs. returning users, ZIP code, neighborhood, time of day, platform, DashPass vs. non-member, cuisine, delivery vs. pickup?
   • How would you guard against confounding, Simpson’s paradox, and selection bias when interpreting the drop?
5. Recommendations
   • If pricing appears to be the main driver, what immediate short-term and longer-term actions would you recommend?
   • What trade-offs would you consider between order volume, profitability, courier earnings, merchant health, and customer experience?
6. Experimentation Propose an experiment or quasi-experiment to test a fix.
   • Define the treatment and control
   • Choose a primary success metric and guardrail metrics
   • Specify the randomization unit, duration, and power/MDE considerations
   • Explain when you would use an A/B test versus a geo experiment or difference-in-differences approach