Design an experiment with marketplace network effects
Overview
This question evaluates a candidate's competency in causal experiment design for networked two‑sided marketplaces, focusing on interference and spillover management, cluster randomization choices, metric definition, power/sample‑size calculations, bias‑reduction approaches, and diagnostic checks.
Uber
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
Causal Experiment Design for a Two‑Sided Marketplace with Interference
You are designing a causal experiment for a new networked product in a two‑sided marketplace (riders and drivers) where interference/network effects are likely (e.g., through matching, wait times, or pricing).
Specify the following:
- Unit of randomization (user, driver, city, or geo‑cluster) and why.
- How you will limit and measure spillovers (e.g., cluster randomization, geographic lift tests, partial‑interference assumptions, graph cuts/holdouts).
- Primary and guardrail metrics and how to compute them.
- Power/sample‑size calculations under clustering (including ICC and design effect) and expected duration.
- Bias‑reduction techniques (e.g., CUPED, pre‑period stratification) and how you’ll handle novelty/washout.
- Diagnostics to detect SUTVA violations and noncompliance.
- How the design changes if driver supply is effectively unlimited.
Provide a concrete rollout plan and analysis outline.
Solution
Login required
Show