Investigate LA Order Drop
Company: DoorDash
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
A food delivery marketplace sees a meaningful decline in completed orders in the Los Angeles market. Explain how you would investigate the drop end to end.
Your answer should:
1. Break the problem into major components such as:
- customer demand
- merchant supply
- courier supply
- product funnel and reliability
- external factors such as seasonality, weather, pricing, promotions, and competitor activity
2. List concrete hypotheses for each component and describe what data you would use to confirm or reject each hypothesis.
3. Decompose the order decline across the funnel, for example:
- app visits or sessions
- search
- menu views
- add-to-cart
- checkout
- order attempts
- completed orders
Also account for marketplace constraints such as merchant online minutes, delivery radius, ETA, stockouts, acceptance rate, cancellations, and operational incidents.
4. Suppose your investigation suggests that the main driver is that merchants are online for fewer minutes per day. The business now wants to test an incentive program intended to increase merchant online time. Design an experiment.
For the experiment design, address:
- the appropriate randomization unit
- why a merchant-level switchback design is not appropriate if a merchant cannot realistically be treated and untreated within the same day
- whether a geo-level or cluster-level holdout is better, and how you would reduce interference
- primary, secondary, and guardrail metrics
- eligibility rules, treatment assignment, and experiment duration
- how to perform power analysis or minimum detectable effect planning
- how you would interpret the result if merchant online minutes increase but completed orders do not
Quick Answer: This question evaluates marketplace analytics, causal inference, hypothesis-driven investigation, funnel decomposition, and experimentation design skills in a data science context.