# Evaluate a Bike Dasher Program with a Controlled Experiment A delivery marketplace is considering a program that encourages some couriers to make deliveries by bicycle rather than by car. The team wants to know whether the program should launch, whether bicycle couriers spend too long waiting near pickup locations, and how to run an experiment when bicycle and car couriers can affect one another in the same market. Design the analysis and experiment. State any assumptions you need instead of inventing operating details. ### Clarifying Questions to Ask - What user, courier, merchant, and marketplace outcomes is the program intended to improve? - Does “parking time” mean arrival-to-pickup dwell time, time spent finding a stopping location, or another interval? - Can couriers switch vehicle type during the test, and can orders be reassigned across treatment groups? - Is rollout possible by courier, geographic zone, time block, or market? ### Part 1: Explain why the program might be worth testing Lay out the causal mechanisms, benefits, risks, and segments that would determine whether bicycle delivery is promising. #### What This Part Should Cover - Testable hypotheses tied to marketplace outcomes - Heterogeneity by trip distance, density, time, weather, or merchant type - Potential courier, customer, merchant, safety, and capacity trade-offs ### Part 2: Decide whether pickup dwell time is too long Define a metric and comparison strategy for pickup dwell time, including how you would handle skew, censoring, and differences in order mix. #### What This Part Should Cover - A precise timestamp-based definition and data-quality checks - Distributional measures rather than only a mean - Fair comparisons through stratification, adjustment, or experimental assignment ### Part 3: Design the launch experiment Specify the hypothesis, treatment, randomization unit, eligibility rules, primary metric, guardrails, duration logic, and decision rule. #### What This Part Should Cover - A randomization unit chosen to match interference risk - Marketplace, courier, customer, and operational metrics - Power, novelty, seasonality, compliance, and intent-to-treat reasoning ### Part 4: Handle contamination between bicycle and car couriers Explain how shared demand, dispatch, incentives, or courier switching could violate standard A/B test assumptions and how you would respond. #### What This Part Should Cover - A concrete interference model - Cluster or switchback alternatives and their trade-offs - Measurement of exposure, spillovers, and compliance ### What a Strong Answer Covers - A decision-oriented causal framework rather than a list of metrics - Precise definitions and validation of operational data - An experiment design that recognizes marketplace interference - A launch rule that balances value, reliability, and guardrails ### Follow-up Questions 1. How would you design the test if only one city were available? 2. What if delivery time improves but courier earnings per active hour fall? 3. How would rain or seasonal demand affect duration and interpretation? 4. Which result would make you stop the experiment early?