How would you test a bike delivery option?

Case: Launch a bike-delivery option

You work on a food delivery marketplace (customers place orders; couriers deliver). The team is considering launching a bike courier delivery option in a dense urban area.

1) Why consider bikes?

Explain the product/business rationale for adding bike couriers (vs only cars/scooters), including when bikes are likely to help and when they may hurt.

2) What should you consider when designing the experiment?

List key factors/risks to account for when running an experiment for bike delivery, such as:

• Marketplace dynamics (courier supply, demand, matching)
• Interference/spillovers (violations of SUTVA)
• Seasonality and external factors (e.g., weather)
• Operational constraints (training, dispatch rules)
• Data quality and logging

3) What metrics would you use?

Propose a metric framework with:

• Primary success metric (one main decision metric)
• Guardrails (safety/quality/cost)
• Diagnostic metrics (to understand mechanisms)

Be explicit about definitions (e.g., what counts as “on-time”, how to treat cancellations, which time window).

4) What experimentation type would you choose?

Recommend an experimentation approach and unit of randomization, and justify your choice. Options may include:

• Classic A/B at order or user level
• Courier-level randomization
• Geo-based test (zone/city)
• Time-based switchback / interleaving

Include how you would handle bias/confounding, estimate sample size/MDE at a high level, and how you would ramp/monitor the launch.