Define market-only rider experience metrics
Overview
This question evaluates a data scientist's competency in designing market-only rider experience metrics and a composite Market Balance Index, testing skills in marketplace metric specification, survival/hazard modeling, price-adjusted indexing, censoring and normalization rules, and experimental backtesting using server-side logs.
Uber
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
0
0
Market-only Rider Experience Metrics and Market Balance Index (MBI)
You are designing a metric suite for a rides marketplace where "rider experience" must reflect market balance (supply–demand dynamics) rather than app UI or vehicle quality. All metrics should be derivable from server-side marketplace logs and be robust to UI changes.
Tasks
-
Define a market-only metric suite that captures rider experience via supply–demand mechanics. Include, at minimum:
- Request Fill Rate within 5 minutes (FR5)
- Time-to-Match (TTM)
- Quoted pickup ETA at request time
- Realized pickup wait
- Rider wait-hazard (survival/hazard-based)
- Price-adjusted wait index
-
Provide precise metric definitions, including:
- Event and unit definitions (what constitutes an intent, request, match, pickup)
- Sampling windows and aggregation levels
- Censoring rules for unfulfilled requests and cancellations
- Holiday/outage exclusions and normalization
- Propose a single composite Market Balance Index (MBI) with an explicit, computable formula and weights across sub-metrics. Explain scaling/normalization so MBI is comparable over markets and time.
- Design a backtest that demonstrates sensitivity of the metric suite and MBI to supply/demand shocks (e.g., weather, events) but insensitivity to UI releases. Describe the study design and report illustrative results.
Solution
Login required
Show