How to experiment on ETA reduction
Company: Uber
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Technical Screen
## Context
You work on a consumer app where users see an **ETA (estimated time to arrival/delivery)** during a funnel (e.g., browsing → checkout → order placed). The team can change product and/or operations to **reduce ETA**.
You have historical data showing a **positive correlation** between ETA and conversion (users are *more* likely to convert when ETA is higher), which seems counterintuitive.
## Questions
1. **Why might reducing ETA be beneficial?** List plausible business/product benefits and what metrics they would affect.
2. **Why might ETA be positively correlated with conversion in observational data?** Provide multiple hypotheses (confounding/selection effects) and how you would test or rule them out.
3. **Design an experiment** to estimate the causal impact of reducing ETA on conversion.
- Define the **treatment**, **control**, and **randomization unit**.
- Specify **primary metric**, **diagnostic metrics**, and **guardrail metrics**.
- Discuss risks like interference, seasonality, and delayed effects.
4. **Confidence intervals:** Explain how you would compute and interpret a confidence interval for the conversion lift.
5. **Unit of analysis:** Why might the analysis **not** be done at the user level? When is user-level appropriate vs session/order/request-level?
Quick Answer: This question evaluates a data scientist's competence in causal inference, A/B test design, metric selection, and diagnosing observational confounding when reducing ETA might affect conversion.