Measure rider incentive causal ROI

Quick Overview

This question evaluates a data scientist's competency in causal inference and experimental design, marketplace health metric construction, ROI estimation, and the handling of selection bias, spillover effects, multiple testing, and heterogeneity in two-sided platforms.

Measure rider incentive causal ROI

Company: Uber

Role: Data Scientist

Category: Statistics & Math

Difficulty: hard

Interview Round: Technical Screen

You plan a rider‑side incentive (e.g., 20% off up to $10) targeted by a propensity model. Estimate causal incrementality and ROI despite selection and marketplace spillovers. Do the following: - Define marketplace‑health metrics with formulas: matching quality (pickup ETA p50, post‑dispatch cancel rate within 5 minutes, driver idle minutes/trip), demand (incremental trips, GMV lift), supply (acceptance rate, earnings/hour), and a composite quality‑of‑match metric (distance‑to‑pickup p50). - Propose identification in three scenarios: (A) randomized geo holdouts; (B) thresholded offer scores enabling regression discontinuity (specify bandwidth selection, continuity checks, McCrary test, and polynomial order); (C) business‑rule targeting enabling IV via an operational friction instrument (state relevance and exclusion, and what threats you’ll test). - Write the ROI formula including cannibalization, subsidy burn, surge/ETA externalities, and habit formation. Specify how to estimate LTV lift and amortize CAC; include confidence intervals via the delta method or bootstrap. - Design a two‑stage randomization (city×week, then rider) to identify direct vs spillover effects; define estimands for total, direct, and indirect effects. - Multiple testing: choose and justify a correction strategy (Bonferroni vs BH vs hierarchical testing) for 1 primary, 3 key secondaries, and ~20 diagnostics. - Heterogeneity: outline a pre‑registered plan to detect effect moderation by city tier and weather using causal forests or group‑wise models while controlling Type‑S/M errors. - Diagnostics: enumerate pre‑trend checks, placebo windows, and negative‑control outcomes; define criteria for declaring success and for rollback.

Quick Answer: This question evaluates a data scientist's competency in causal inference and experimental design, marketplace health metric construction, ROI estimation, and the handling of selection bias, spillover effects, multiple testing, and heterogeneity in two-sided platforms.

|Home/Statistics & Math/Uber
Uber logo
Uber
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenStatistics & Math
6
0

Rider Incentive Targeting: Causal Incrementality, ROI, and Spillovers

Context: You plan a rider‑side incentive (e.g., “20% off up to $10”) targeted by a propensity model. You must estimate causal incrementality and ROI in a two‑sided marketplace with selection and spillovers.

Do the following:

1) Marketplace‑Health Metrics (with formulas)

Define and provide formulas for:

  • Matching quality: pickup ETA p50, post‑dispatch cancel rate within 5 minutes, driver idle minutes/trip.
  • Demand: incremental trips, GMV lift.
  • Supply: acceptance rate, earnings/hour.
  • Composite quality‑of‑match metric: distance‑to‑pickup p50.

2) Identification Under Three Scenarios

Propose identification strategies for:

  • (A) Randomized geo holdouts.
  • (B) Thresholded offer scores enabling regression discontinuity: specify bandwidth selection, continuity checks, McCrary test for manipulation, and polynomial order.
  • (C) Business‑rule targeting enabling IV via an operational friction instrument: state relevance and exclusion, and list threats you will test.

3) ROI Formula and Estimation

Write the ROI formula including cannibalization, subsidy burn, surge/ETA externalities, and habit formation. Specify how to estimate LTV lift and amortize CAC. Include confidence intervals via the delta method or bootstrap.

4) Two‑Stage Randomization for Spillovers

Design a two‑stage randomization (city×week, then rider) to identify direct vs spillover effects, and define estimands for total, direct, and indirect effects.

5) Multiple Testing

Choose and justify a correction strategy (Bonferroni vs BH vs hierarchical testing) for: 1 primary, 3 key secondaries, and ~20 diagnostics.

6) Heterogeneity

Outline a pre‑registered plan to detect effect moderation by city tier and weather using causal forests or group‑wise models while controlling Type‑S/M errors.

7) Diagnostics and Decisioning

Enumerate pre‑trend checks, placebo windows, and negative‑control outcomes. Define criteria for declaring success and for rollback.

Loading comments...