Evaluating and Rolling Out Faster Deliveries That Cost More in a Delivery Marketplace

A food-delivery marketplace connects customers, merchants such as restaurants, and couriers. It is considering a change that would shorten delivery times but would cost more money. Would you recommend making the change, and how would you decide? Then explain how you would roll it out. The report gives only this premise and says the follow-up questions went into the details of the rollout. It does not say what the change is, how large the time savings or the extra cost are, or who pays, so treat those as clarifying questions rather than assuming them. ### Clarifying Questions - What is the change: paying couriers more, adding courier capacity, sending orders on their own instead of batching several on one trip, or an optional faster-delivery tier? - Who bears the extra cost: the platform, customers through a fee, or merchants? - Does it apply to every order, or only to some markets, times of day or customers? - How much faster and how much more expensive is each order, and how much do both vary by market and time of day? - What is the goal: order growth, customer retention, or better customer experience within a cost limit? ### Part 1 — Decide whether faster delivery is worth the cost Explain how you would decide whether the faster delivery is worth its extra cost: the framework, the metrics, and the analysis or experiment you would use. ```hint What a minute buys Ask how a shorter delivery changes what customers do on later orders, not only how they feel about this one. ``` ```hint The couriers are shared Faster delivery for some orders may come from couriers who would otherwise serve other orders nearby. Ask what that does to a comparison between groups. ``` #### What This Part Should Cover - A value-versus-cost framework over a horizon long enough for retention effects - Metrics for customers, couriers and merchants, with a decision metric and guardrails - An experiment design that handles shared courier supply, and how results may differ across markets and segments ### Part 2 — Roll it out Suppose the evidence supports launching. How would you roll the change out, and what would make you pause or reverse it? ```hint Every rollout step is a measurement Ask how the order in which markets switch over could keep giving you an estimate of the effect. ``` ```hint Decide the stop rules early Write down, before the first market switches, which numbers would pause the rollout. ``` #### What This Part Should Cover - Staging by market, time and exposure, and what each stage must show before the next - Monitoring, guardrail thresholds and rollback - A long-term holdout, and when the policy should be targeted rather than universal ### What a Strong Answer Covers - Clarifies what the change is and who pays before analyzing it - Converts faster delivery into incremental orders and retention that can be compared with the added cost per order - Two-sided marketplace thinking that covers customers, couriers and merchants - An interference-aware experiment design, including its trade-off with measuring long-term retention - A launch decision rule, segment-level targeting, and a staged rollout with explicit guardrails ### Follow-up Questions - If treated customers get faster deliveries partly by taking couriers away from control customers, how does that bias a customer-randomized test, and what design would you use instead? - Order frequency rises in the first two weeks after launch, then the effect fades. How do you tell novelty from a lasting change? - If the faster delivery is sold as an optional paid tier, how would you set its price? - The change pays for itself in dense cities but not in suburbs. What do you recommend?