Design analysis to reduce cold-delivery complaints
Company: DoorDash
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
You are the data scientist at a food-delivery marketplace seeing an increase in 'cold food' complaints. Design an end-to-end analysis and action plan (no product/app UI changes allowed). Be precise: 1) Define one primary metric and at least two guardrails. Specify exact formulae, numerator/denominator, inclusion/exclusion rules (completed deliveries only; first cold complaint within 24h per order; exclude re-deliveries/cancellations), time window, and aggregation grain. Justify why your primary metric is superior to alternatives (e.g., refund rate, mean drop-off temperature, CSAT). 2) Enumerate the key slices and covariates (city, weather, cuisine, container type, distance, elevation, pickup wait, drop-off wait, courier equipment such as insulated bag, batching, time-of-day, weekend/holiday, restaurant prep SLAs). 3) Lay out a diagnostic plan to attribute drivers: separate kitchen- vs. courier- vs. handoff-driven cooling using stage-duration breakdowns and appropriate models; describe controls for confounding (seasonality, surge, rain) and small-sample shrinkage. 4) Propose an experiment to provide couriers with insulated bags: state randomization unit, stratification, sample-size/power back-of-envelope with assumptions, duration, primary and secondary metrics, and spillover/contamination risks. 5) If an RCT is infeasible, propose a quasi-experiment (e.g., difference-in-differences on staggered rollouts, or IV using sudden rain as an instrument). State identifying assumptions and two falsification checks. 6) Recommend 3–5 operational changes that do not alter the product (e.g., operational batching limits, pickup queue prioritization, courier incentives for swift handoff, restaurant packaging standards), and how you would measure impact at 2 weeks vs. 8 weeks, including ROI and operational cost trade-offs. Assume 'today' is 2025-09-01.
Quick Answer: This question evaluates a data scientist's end-to-end analytics and experimentation competencies, including precise metric definition, causal diagnostics and attribution, randomized and quasi-experimental design, and operational impact measurement.