Investigate Falling Successful Orders

Quick Overview

This question evaluates a data scientist's ability to diagnose a drop in a product metric through anomaly verification, data validity checks, funnel and cohort analysis, causal attribution across marketplace sides (customer, restaurant, delivery), and experiment design.

Investigate Falling Successful Orders

Company: DoorDash

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

You are interviewing for a Data Scientist role at DoorDash. In the Los Angeles market, the metric `successful orders per day` has declined over the last 2 weeks relative to the previous 4-week baseline. Assume: - A successful order is an order that is placed by a customer and ultimately delivered successfully without cancellation. - Metrics are aggregated by calendar day in the `America/Los_Angeles` timezone. - You have access to customer app session logs, checkout events, pricing and ETA estimates, restaurant availability and acceptance logs, order cancellations, dasher supply and dispatch events, promotions, outages, weather, and geographic metadata. How would you investigate this drop? 1. First explain how you would verify that the decline is real and not caused by data issues, seasonality, or mix shifts. 2. Give one plausible hypothesis from each side of the marketplace: customer, restaurant, and dasher. 3. If all three hypotheses appear to be contributing, how would you quantify the impact of each one on the decline in successful orders? 4. If you use a funnel-based approach, what funnel would you build, which metrics would you inspect at each step, and how would you isolate where the largest drop occurs? 5. Based on the likely root causes, propose one or more product or operational improvements. 6. Pick one improvement and design an A/B test. Specify the unit of randomization, primary metric, guardrail metrics, experiment duration, and any interference, bias, or power issues you would consider.

Quick Answer: This question evaluates a data scientist's ability to diagnose a drop in a product metric through anomaly verification, data validity checks, funnel and cohort analysis, causal attribution across marketplace sides (customer, restaurant, delivery), and experiment design.

|Home/Analytics & Experimentation/DoorDash
DoorDash logo
DoorDash
Nov 1, 2025, 12:00 AM
hardData ScientistTechnical ScreenAnalytics & Experimentation
7
0

You are interviewing for a Data Scientist role at DoorDash. In the Los Angeles market, the metric successful orders per day has declined over the last 2 weeks relative to the previous 4-week baseline.

Assume:

  • A successful order is an order that is placed by a customer and ultimately delivered successfully without cancellation.
  • Metrics are aggregated by calendar day in the America/Los_Angeles timezone.
  • You have access to customer app session logs, checkout events, pricing and ETA estimates, restaurant availability and acceptance logs, order cancellations, dasher supply and dispatch events, promotions, outages, weather, and geographic metadata.

How would you investigate this drop?

  1. First explain how you would verify that the decline is real and not caused by data issues, seasonality, or mix shifts.
  2. Give one plausible hypothesis from each side of the marketplace: customer, restaurant, and dasher.
  3. If all three hypotheses appear to be contributing, how would you quantify the impact of each one on the decline in successful orders?
  4. If you use a funnel-based approach, what funnel would you build, which metrics would you inspect at each step, and how would you isolate where the largest drop occurs?
  5. Based on the likely root causes, propose one or more product or operational improvements.
  6. Pick one improvement and design an A/B test. Specify the unit of randomization, primary metric, guardrail metrics, experiment duration, and any interference, bias, or power issues you would consider.
Loading comments...