Design experiment for bike delivery feature

Quick Overview

Evaluates experimental design, causal inference, metric selection, and statistical power with attention to interference and operational constraints, categorized under Analytics & Experimentation and scoped to an applied, mid-to-senior data scientist abstraction level.

Design experiment for bike delivery feature

Company: DoorDash

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

You work on a delivery marketplace (customers, merchants, couriers). The company is considering launching a **“bike delivery”** capability in a subset of cities. The product idea: in eligible areas, the system can match some orders to bike couriers (or show customers a “bike delivery” option) to improve delivery efficiency and/or reduce costs. However, bike delivery could also change delivery times, cancellation rates, courier supply, and customer satisfaction. ### Prompt Design an experiment (or quasi-experiment) to evaluate whether enabling bike delivery is beneficial. In your answer, cover: 1) **Goal and decision**: what decision will the experiment inform? 2) **Primary metric** and why it best matches the goal. 3) **Secondary / guardrail metrics** (pick a few) and how they trade off. 4) **Unit of randomization** (user, order, courier, geo) and rationale; discuss interference/network effects. 5) **Experiment design**: treatment/control definition, eligibility, ramp strategy, duration. 6) **Key biases/confounders** to watch (selection, seasonality, supply constraints, Simpson’s paradox, etc.) and mitigations. 7) **Analysis plan**: how you would estimate impact (ITT vs TOT), handle non-compliance, and check heterogeneity. 8) **Power/MDE**: what inputs you need to size the test and how you’d think about MDE.

Quick Answer: Evaluates experimental design, causal inference, metric selection, and statistical power with attention to interference and operational constraints, categorized under Analytics & Experimentation and scoped to an applied, mid-to-senior data scientist abstraction level.

|Home/Analytics & Experimentation/DoorDash
DoorDash logo
DoorDash
Nov 15, 2025, 12:00 AM
mediumData ScientistTechnical ScreenAnalytics & Experimentation
5
0

You work on a delivery marketplace (customers, merchants, couriers). The company is considering launching a “bike delivery” capability in a subset of cities.

The product idea: in eligible areas, the system can match some orders to bike couriers (or show customers a “bike delivery” option) to improve delivery efficiency and/or reduce costs. However, bike delivery could also change delivery times, cancellation rates, courier supply, and customer satisfaction.

Prompt

Design an experiment (or quasi-experiment) to evaluate whether enabling bike delivery is beneficial.

In your answer, cover:

  1. Goal and decision : what decision will the experiment inform?
  2. Primary metric and why it best matches the goal.
  3. Secondary / guardrail metrics (pick a few) and how they trade off.
  4. Unit of randomization (user, order, courier, geo) and rationale; discuss interference/network effects.
  5. Experiment design : treatment/control definition, eligibility, ramp strategy, duration.
  6. Key biases/confounders to watch (selection, seasonality, supply constraints, Simpson’s paradox, etc.) and mitigations.
  7. Analysis plan : how you would estimate impact (ITT vs TOT), handle non-compliance, and check heterogeneity.
  8. Power/MDE : what inputs you need to size the test and how you’d think about MDE.
Loading comments...