Cold Food Complaints: Metrics, Diagnosis, and Experiment Design

Context and assumptions:

• You are analyzing a spike in “food arrived cold” complaints at a large food-delivery platform.
• Temperature sensors are not available; you will rely on operational logs, GPS, weather data, and complaint labels.
• You have order-level event timestamps (order placed, merchant accepts, ready, courier assigned/arrives/picks up, drop-off), GPS traces, distance, batching metadata, cuisine/packaging, courier bag verification flags, refunds, and CSAT/NPS.

Tasks

1. Define the core outcome metric(s) and build a metric tree to isolate where heat loss likely occurs (e.g., prep_time, wait_at_restaurant, transit_time, distance, batching_count, outside_temp, packaging_type, courier_bag_type). For each metric, specify how you will measure it and the guardrails you will monitor (e.g., ETA accuracy, cancellations, CSAT/NPS, refund_rate).
2. Propose a structured diagnostic plan (within 72 hours) to prioritize highest-variance contributors. Specify the slices, cohorts, and negative controls (e.g., wrong_item complaints as a negative control, weather-matched day-over-day, restaurant fixed effects).
3. Design one decisive A/B test to reduce cold deliveries (e.g., mandate insulated bags for a subset of couriers or disable batching beyond 2 orders for long distances). Specify: experimental unit and randomization (e.g., courier-day, restaurant-day), sample-size assumptions (baseline cold_rate and minimal detectable effect), primary/secondary endpoints, guardrails, power, duration, ramp plan, and spillover mitigation.
4. Explain how you’d attribute improvements to the change vs. concurrent factors like weather or promotions (e.g., difference-in-differences city pairs, CUPED, or stratified randomization).
5. If the test increases delivery time by 6% but lowers cold_rate by 2 percentage points, outline a decision framework to trade off customer experience vs. speed, and specify follow-up tests you’d run.