Design experiment for bike delivery feature
Company: DoorDash
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You work on a delivery marketplace (customers, merchants, couriers). The company is considering launching a **“bike delivery”** capability in a subset of cities.
The product idea: in eligible areas, the system can match some orders to bike couriers (or show customers a “bike delivery” option) to improve delivery efficiency and/or reduce costs. However, bike delivery could also change delivery times, cancellation rates, courier supply, and customer satisfaction.
### Prompt
Design an experiment (or quasi-experiment) to evaluate whether enabling bike delivery is beneficial.
In your answer, cover:
1) **Goal and decision**: what decision will the experiment inform?
2) **Primary metric** and why it best matches the goal.
3) **Secondary / guardrail metrics** (pick a few) and how they trade off.
4) **Unit of randomization** (user, order, courier, geo) and rationale; discuss interference/network effects.
5) **Experiment design**: treatment/control definition, eligibility, ramp strategy, duration.
6) **Key biases/confounders** to watch (selection, seasonality, supply constraints, Simpson’s paradox, etc.) and mitigations.
7) **Analysis plan**: how you would estimate impact (ITT vs TOT), handle non-compliance, and check heterogeneity.
8) **Power/MDE**: what inputs you need to size the test and how you’d think about MDE.
Quick Answer: Evaluates experimental design, causal inference, metric selection, and statistical power with attention to interference and operational constraints, categorized under Analytics & Experimentation and scoped to an applied, mid-to-senior data scientist abstraction level.