Design station experiment with interference and rush-hour spillovers
Company: Uber
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
NYC’s Penn Station is piloting a new in-station ordering product with two competing features A and B; foot traffic is highly periodic (weekday rush hours), users may share networks/devices, and spatial/temporal spillovers are strong. How would you determine which feature is better? Choose and justify an experimental design under interference—station-level switchback (A/B by time) versus geo-split versus synthetic control—and specify the randomization unit, period length, washout, and guardrails to limit contamination; define primary KPIs and guardrails (e.g., conversion, throughput/minute, queue abandonment, AOV, CSAT) and a decision rule, including MDE and power assumptions; explain how you would handle shocks like service disruptions and non-stationarity, construct a synthetic control (donor pool, pre-period fit, placebo tests), and combine it with switchback results; outline diagnostics for carryover, novelty, and day-of-week effects, and describe variance reduction (pairing, CUPED) and how you would estimate heterogeneous lift by time block to confidently select the winning feature.
Quick Answer: This question evaluates a data scientist's competency in experimental design and causal inference under interference and non-stationarity, covering skills such as randomization unit and period selection, KPI and guardrail specification, synthetic control construction, variance-reduction techniques, and diagnostics for spillovers and temporal effects. It is commonly asked in the Analytics & Experimentation domain to assess the ability to design robust, real-world experiments and is primarily a practical application that also requires strong conceptual understanding of bias, carryover, seasonality, shocks, and validation methods.