Diagnose completed orders drop in Los Angeles

You are a data scientist at DoorDash supporting the consumer pricing team. The number of **completed delivery orders in Los Angeles** has dropped meaningfully over the last two weeks relative to both historical baseline and similar large cities. How would you investigate the issue end-to-end? In your answer, cover: 1. **Problem framing and validation** - How would you confirm the drop is real rather than seasonality, reporting latency, instrumentation bugs, or a temporary outage? - What benchmarks would you use: week-over-week, year-over-year, pre/post launch, and comparison to matched control cities? 2. **Metrics and funnel decomposition** Be explicit about the metrics you would examine and how they relate to completed orders. For example: - **Demand**: app opens, sessions, store views, checkout starts, order attempts - **Pricing and conversion**: basket size, menu price index, delivery fee, service fee, surge or small-order fees, promotions, DashPass/member mix, checkout conversion - **Marketplace health**: merchant availability, out-of-stock rate, merchant acceptance rate, courier supply, assignment time, ETA, cancellations - **Outcome metrics**: completed orders, completed orders per active consumer, gross order value, contribution margin 3. **Hypotheses** Generate and prioritize plausible explanations, including: - a pricing change that reduced conversion - lower courier supply or higher ETAs - merchant outages or lower assortment availability - app/checkout product regressions - external factors such as weather, major events, regulation, or competitor promotions - mix shift across neighborhoods, user cohorts, or dayparts 4. **Segmentation and causal reasoning** - How would you segment the analysis: new vs. returning users, ZIP code, neighborhood, time of day, platform, DashPass vs. non-member, cuisine, delivery vs. pickup? - How would you guard against confounding, Simpson’s paradox, and selection bias when interpreting the drop? 5. **Recommendations** - If pricing appears to be the main driver, what immediate short-term and longer-term actions would you recommend? - What trade-offs would you consider between order volume, profitability, courier earnings, merchant health, and customer experience? 6. **Experimentation** Propose an experiment or quasi-experiment to test a fix. - Define the treatment and control - Choose a primary success metric and guardrail metrics - Specify the randomization unit, duration, and power/MDE considerations - Explain when you would use an A/B test versus a geo experiment or difference-in-differences approach