Experiment Design: Surge-Cap Algorithm for NYC (20:00–22:00)

Context

A new surge-cap algorithm is proposed to improve rider trip completion in NYC during 20:00–22:00 without materially harming ETA or neighborhood fairness. Pricing operates on 250 geo-cells.

Baseline (NYC, 20:00–22:00)

• Requests per evening ≈ 500,000; completion rate = 82%.
• Average ETA = 6.5 minutes (SD across geo-cells = 2.0 minutes).
• Pricing units: 250 geo-cells; average requests per cell per evening ≈ 2,000.
• Coefficient of variation of requests across cells ≈ 0.6.
• Intra-cell intraclass correlation (ICC) for completion ≈ 0.05.

Tasks

1. Choose an experimental design (e.g., rider-level randomization, driver-level randomization, geo switchback by time, or city-level holdout). Justify your choice considering interference/spillovers in pricing and supply.
2. Define the primary metric, directional hypothesis, and guardrails (ETA, driver cancellations, demand spillover to adjacent hours, and neighborhood fairness across income quartiles). Specify exact uplift and guardrail thresholds that would trigger a stop.
3. Power and duration: Target a minimum detectable effect (MDE) of +0.8 percentage points (pp) in completion. Using the provided aggregation structure (geo-cell as the unit) and ICC, approximate the number of switchback days needed to reach 80% power at α = 0.05. State all assumptions (variance model, independence between alternating periods, balancing by weekday) and show the formula you use.
4. Diagnostics: Describe how to run parallel-trend checks using pre-period data and a difference-in-differences (DiD) estimator with an external control market (e.g., Chicago). Specify the DiD regression and key robustness checks (placebo tests, event study, cluster-robust SEs).
5. Implementation: Outline assignment, logging, and exposure rules to prevent cross-over contamination for riders and drivers during the test window.