##### Scenario DoorDash, a food-delivery marketplace, is seeing growing customer complaints about orders arriving late. You are the data scientist asked to diagnose the problem and recommend a fix. ##### Question Diagnose the root causes of delivery delays and design a validation experiment. 1. **Define the problem.** What does "late" mean here, and what is your primary KPI (e.g., on-time delivery rate) versus your secondary / guardrail metrics? Why? 2. **Metrics to examine first.** Which delivery-performance metrics would you look at first, and how do you instrument the order lifecycle so you can localize where the delay happens? 3. **Identify root causes.** How would you attribute excess delay to specific stages (assignment, courier travel, restaurant prep, pickup dwell, drop-off travel) and form hypotheses (courier supply vs. demand, prep-time underestimation, dispatch/batching logic, geography, time-of-day)? 4. **Segment the problem.** Which segments (region/zone, restaurant cohort, courier supply, order attributes, time/weather) would you cut by to localize the issue and prioritize? 5. **Design an experiment / product change.** Propose one solution and design an A/B or geo-holdout test to validate it: unit of randomization, primary outcome, guardrails, duration, power/sample size, and analysis plan. Address interference between nearby zones. ##### Hints Clarify the lateness definition (initial promise vs. latest ETA) to avoid ETA-padding gaming. Decompose end-to-end time into stages and compare each stage to an expected baseline. Segment by region/restaurant/courier/time. Because courier supply is shared across nearby areas, prefer cluster (geo-zone) randomization over per-order randomization, and design for spillovers.

This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Identify Key Metrics to Address Delivery Delays states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at DoorDash.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at DoorDash during technical interviews.

Identify Key Metrics to Address Delivery Delays

Scenario

DoorDash, a food-delivery marketplace, is seeing growing customer complaints about orders arriving late. You are the data scientist asked to diagnose the problem and recommend a fix.

Question

Diagnose the root causes of delivery delays and design a validation experiment.

Define the problem. What does "late" mean here, and what is your primary KPI (e.g., on-time delivery rate) versus your secondary / guardrail metrics? Why?
Metrics to examine first. Which delivery-performance metrics would you look at first, and how do you instrument the order lifecycle so you can localize where the delay happens?
Identify root causes. How would you attribute excess delay to specific stages (assignment, courier travel, restaurant prep, pickup dwell, drop-off travel) and form hypotheses (courier supply vs. demand, prep-time underestimation, dispatch/batching logic, geography, time-of-day)?
Segment the problem. Which segments (region/zone, restaurant cohort, courier supply, order attributes, time/weather) would you cut by to localize the issue and prioritize?
Design an experiment / product change. Propose one solution and design an A/B or geo-holdout test to validate it: unit of randomization, primary outcome, guardrails, duration, power/sample size, and analysis plan. Address interference between nearby zones.

Hints

Clarify the lateness definition (initial promise vs. latest ETA) to avoid ETA-padding gaming. Decompose end-to-end time into stages and compare each stage to an expected baseline. Segment by region/restaurant/courier/time. Because courier supply is shared across nearby areas, prefer cluster (geo-zone) randomization over per-order randomization, and design for spillovers.

Constraints & Assumptions

Preserve the scope, facts, inputs, and requested outputs from the prompt above.
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask

Clarify the business objective, unit of analysis, time window, exposure definition, and primary metric.
State assumptions about instrumentation, randomization, sample size, and data quality.
Separate descriptive analysis from causal claims.

What a Strong Answer Covers

A metric framework with primary, guardrail, and diagnostic metrics.
A credible analysis or experiment design with clear assumptions and bias checks.
SQL/statistical logic for segmentation, variance, confidence, and data validation where relevant.
An actionable recommendation that explains trade-offs and next steps.

Follow-up Questions

What sanity checks would you run before trusting the result?
How would you handle novelty effects, seasonality, or selection bias?
What decision would you make if metrics disagree?