Design an RCT for app-open discount
Company: Uber
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
Your company plans a 'X dollars off on app open' promotion in a two‑sided marketplace. Design a randomized controlled experiment to test success: (1) pick one primary success metric aligned to long‑term profit (e.g., 28‑day incremental gross profit per user) and list guardrails (refund rate, support tickets, seller cancellations, cannibalization of full‑price orders); (2) choose and justify the unit of randomization (user vs device vs account vs session) considering repeated sessions per user, cross‑device logins, households sharing devices, and seller‑side spillovers; (3) specify exposure/eligibility rules, sticky assignment, holdout enforcement, and contamination controls; (4) describe how you will detect/mitigate network effects (geo or cluster randomization, graph clustering, exposure mappings) and how you’d quantify spillovers; (5) provide sample size and test duration with assumptions and MDE, ramp and throttle plan, and stopping rules; (6) list instrumentation/logging needed to compute ITT and TOT (e.g., assignment id, exposure timestamp, eligibility flags, device id, session id).
Quick Answer: This question evaluates a data scientist's competency in experimental design and causal inference, including metric definition, statistical power/sample-size reasoning, instrumentation, and managing interference and spillovers in two-sided marketplaces.