Experimentation: Evaluate a “Top Dasher” incentive program
A delivery platform wants to launch or change a Top Dasher program (e.g., priority access to orders, higher ranking, or incentives) intended to improve reliability and coverage.
However, this is a marketplace with strong interference/spillovers: if some couriers get priority, other couriers and customers may be affected.
Task
-
Define the
goal
of the program and propose a
metric tree
:
-
Primary outcome metric(s)
-
Diagnostic metrics
-
Guardrails
-
Choose an
experimental design
and justify the
randomization unit
(examples: courier, courier-day, zone, zone-time).
-
Discuss when to use:
-
Individual randomization
-
Cluster randomization
-
Switchback
experiments
-
Clustered analysis / variance estimation
-
If using a
switchback
design, list the main threats (carryover, noncompliance, seasonality, novelty) and how you would mitigate them.
-
Explain how you would reason about
false positives vs false negatives
(Type I vs Type II error) for this decision, including which risk is worse and why.
Output
Provide a concrete recommended design (including analysis approach) and how you would communicate trade-offs to stakeholders.