Measure Impact of Updated Rider ETA Algorithm
Overview
This question evaluates a data scientist's competency in experimental design, causal inference, metric definition, and marketplace analytics for measuring how an ETA model update impacts demand and supply.
Uber
Jul 12, 2025, 6:59 PM
Data Scientist
Technical Screen
Analytics & Experimentation
43
0
Scenario
A ride-hailing company has updated its rider ETA-prediction algorithm (the ETA shown to riders before they request a trip) and wants to quantify the marketplace impact on both demand (riders) and supply (drivers).
Task
Design an experiment to measure how the new rider-ETA model affects the business.
Questions
- Which primary success metrics would you track on the demand and supply sides?
- What intermediate or leading indicators would you monitor?
- Would you pick a classic A/B test or a synthetic-control approach? Explain the design, randomization unit, duration, and how you would interpret results.
Hints
- Map the customer → marketplace funnel; define guardrail KPIs.
- Consider network interference, seasonality, and statistical power before choosing between parallel A/B and geo-level synthetic control.
Solution
Login required
Show