## Design and Evaluate a Refund Policy for Delayed Orders A delivery marketplace is considering automatically refunding customers whose orders are delayed. Propose a policy and an evaluation plan that balance customer recovery with marketplace cost and unintended behavior. ### Constraints & Assumptions - “Delayed” must be defined relative to a promised or estimated delivery time, not only by total duration. - A refund can affect customer trust, future ordering, support contacts, merchant outcomes, and platform cost. - Some delays are foreseeable before delivery, while others are only known after the order is completed. - The policy must avoid systematically treating similar customers differently without a defensible reason. ### Clarifying Questions to Ask - Is the goal to retain customers, reduce support burden, improve perceived fairness, or meet an existing service promise? - Does “refund” mean a full refund, partial refund, credit, or reimbursement of a fee? - How accurate are promised-time estimates, and have their definitions changed? - Who bears the cost, and are there legal, contractual, or abuse-prevention constraints? - Can eligibility and refund amount be randomized during a test? ### What a Strong Answer Covers - Defines delay severity and policy eligibility using a stable order-level denominator. - Chooses a primary decision metric and customer, courier, merchant, and financial guardrails. - Separates policy value from errors in the promised-time model. - Proposes threshold, amount, and communication rules that can be evaluated incrementally. - Uses a randomized or defensible phased design and accounts for delayed outcomes such as retention. - Addresses heterogeneous effects, abuse risk, fairness, operational feasibility, and stopping criteria. ### Follow-up Questions 1. How would you choose between cash refunds and platform credits? 2. What if the policy improves retention but loses money over the measured horizon? 3. How would you detect customers or merchants adapting strategically to the threshold? 4. How would you evaluate a policy that cannot be randomized for operational reasons?