Design marketplace experiments at DoorDash
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You are interviewing for a product data role at DoorDash. Consider the following marketplace scenarios.
1. **Top Dasher program**
DoorDash runs a status program for high-performing Dashers (for example, better access to scheduling or order opportunities).
- What are the main product and marketplace pros and cons of such a program?
- How would you define success metrics? Include a primary metric, supporting metrics, and guardrails across the consumer, dasher, and merchant sides of the marketplace.
- What should the randomization unit be for an experiment: dasher, order, market, or time-based switchback? Explain the trade-offs and any interference concerns.
- Suppose the treatment group's key metric is lower than the control group's. How would you investigate whether this is a true negative impact versus a data quality issue, power issue, imbalance, novelty effect, or spillover?
2. **Order cancellation rate has increased**
DoorDash observes that overall order cancellation rate is materially higher than usual.
- Which teams or parts of the organization are affected?
- How would you diagnose the increase? Build a structured analysis plan to identify root causes.
- What hypotheses would you test, and how would you validate them?
- In your answer, consider funnel stage, who initiated the cancellation, market and time segmentation, merchant quality, inventory availability, delivery ETA, dasher supply, weather, app performance, and recent policy or product changes.
3. **Merchant self-serve promotions vs. automatic promotions**
DoorDash is deciding between two systems: merchants configure promotions manually, or DoorDash automatically recommends or sets up promotions for them.
- What are the pros and cons of each approach?
- How would you design an experiment to compare them?
- What randomization unit would you choose, and why?
- What trade-offs would you watch for, such as adoption, merchant trust, cannibalization, incremental GMV, contribution margin, customer discount dependency, and heterogeneous effects by merchant type or market?
Quick Answer: This question evaluates a candidate's skills in experimental design, product and marketplace analytics, causal inference, and diagnostic data science for multi-sided platforms.