# Evaluate a Bike-Delivery Pilot with Marketplace Metrics A delivery marketplace is considering a bike-based delivery option. Decide why the business might launch it, choose an initial city, measure how long car couriers spend parking, and design an experiment comparing the new mode with the current delivery system. ### Constraints & Assumptions - Cities differ in density, weather, trip distance, courier supply, and traffic regulation. - Courier mode cannot be assumed random because people self-select vehicles and work areas. - Parking time is not directly logged and may require a proxy or a dedicated measurement study. - Customer, courier, merchant, and marketplace outcomes can move in different directions. ### Clarifying Questions to Ask - Is the goal lower cost, faster delivery, better reliability, lower emissions, or added capacity? - Which orders are physically eligible for bike delivery? - Can mode assignment be randomized within a city, or is geographic rollout the only option? ### Part 1 — Launch rationale and city choice Define the value proposition and construct a city-selection scorecard using demand, distance, density, courier supply, operational feasibility, and downside risk. #### What This Part Should Cover - A causal business hypothesis - Eligibility and capacity estimates - A scorecard with measurable inputs and explicit weights ### Part 2 — Parking-time measurement Define car-courier parking time and propose one primary measurement method plus a validation sample. #### What This Part Should Cover - Clear start and end events - Bias analysis for GPS or app-state proxies - Ground-truth validation and privacy constraints ### Part 3 — Experiment Choose a randomization unit, metrics, guardrails, duration, and analysis plan that handles interference and courier learning. #### What This Part Should Cover - Treatment eligibility and compliance - Marketplace-wide and participant metrics - Spillover, power, novelty, and ramp considerations ### What a Strong Answer Covers - A decision-linked primary metric - Metrics for all marketplace sides - A credible identification strategy ```hint Map the marketplace effects A faster treated delivery can change courier availability and wait times for untreated orders. Include those equilibrium effects in both randomization and analysis. ``` ### Follow-up Questions - How would you proceed if randomization is infeasible? - What result would make you stop the pilot despite faster deliveries?