LA Dinner-Period Orders Down 12% WoW: Diagnose and Validate Root Cause

Context

You are analyzing a weekly decline in a two-sided delivery marketplace. For the week of 2025-08-25 vs. week of 2025-08-18 in Los Angeles:

• Completed orders: −12% WoW
• Early indicators: sessions +4% WoW, order creations +1% WoW
• Accept rate: 94% → 88%
• p95 ETA: 42 → 58 minutes
• Cancellation (reason = "no_courier"): 1.5% → 5.8%
• Restaurant availability, pricing, and marketing spend: unchanged
• Most of the decline occurred 18:00–21:00 local time

Task

Design a rigorous investigation and test plan that cleanly separates demand-side from supply-side causes and validates the true driver.

A) Diagnosis

Lay out a funnel from session → order_created → accepted → picked_up → completed. Specify:

1. Exact cuts: by hour, ZIP, courier density, restaurant category, distance bands, and surge level.
2. Sanity checks for tracking/config regressions.
3. The minimal set of plots/tables to prove whether supply is the bottleneck.
4. Additional metrics you would compute, and thresholds that would convince you.

B) Causal Validation (Experiment)

Assuming supply shortage during the dinner peak is your leading hypothesis, propose an experiment to fix it (e.g., targeted courier incentives or pricing changes). Define:

• Experimental unit and randomization (e.g., ZIP clusters within LA with interference mitigation)
• Primary and guardrail metrics
• Success criteria
• Power/MDE assumptions, sample size, and duration
• Ramp plan
• How you will address seasonality and spillovers

C) Quasi-Experimental Alternative

If an RCT is infeasible, specify a quasi-experimental alternative (e.g., difference-in-differences or synthetic control using other SoCal metros), including pre-trend checks and robustness tests. Provide a concrete analysis checklist for either path so another analyst can execute without ambiguity.