How should Uber evaluate lower ETA?

Q: How should Uber evaluate lower ETA?

This question evaluates competency in metrics design, causal inference, and experiment strategy for two-sided marketplaces, including detecting and addressing interference, interpreting observational correlations versus causal effects, and understanding statistical inference and subgroup selection bias.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Uber.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Uber during technical interviews.

Question

Loading...

Uber wants to estimate the business value of reducing pickup ETA, where ETA refers to the rider-facing estimated time for a driver to arrive at the pickup location. Assume the change may affect both rider behavior and marketplace dynamics, so interference between riders and drivers is possible.

Answer the following:

What metrics would you track to evaluate the impact of lowering ETA?
- Include primary success metrics, downstream business metrics, guardrail metrics, and diagnostic metrics.
- Consider rider outcomes, driver outcomes, marketplace health, and financial impact.
From a business perspective, why is lowering ETA important?
- Discuss how ETA may affect conversion, cancellations, retention, supply-demand balance, and user trust.
- Also discuss possible tradeoffs or unintended consequences.
Suppose that in observational data you find a positive correlation between ETA and session conversion rate, where session conversion rate is defined as:
- successful trip requests / app-open sessions
How would you interpret this result?
- Why does correlation not necessarily imply a causal effect?
- What confounding variables, selection effects, or aggregation effects could produce this pattern?
How would you design a causal evaluation of lowering ETA?
- Compare when you would use a user-level A/B test, a geo-time switchback experiment, or a synthetic control approach.
- Explain the experimental unit, randomization strategy, analysis plan, and how you would handle marketplace spillovers.
After the experiment, suppose the 95% confidence interval for the treatment effect on session conversion rate is [-5%, +1%] .
- How would you interpret this result statistically and from a business decision perspective?
- What does it say about uncertainty, power, and practical significance?
- How would you improve precision in a future experiment?
Now assume the experiment was run as a user-level A/B test. During readout, the PM asks for a deep dive on users who took at least 5 trips during the experiment.
- How would you respond?
- What is problematic about conditioning on this subgroup after randomization?
- How would you communicate the risk of post-treatment bias or selection bias, and what alternative analyses would you recommend instead?

How should Uber evaluate lower ETA?

Quick Overview

Solution

Comments (0)