Evaluate Smart Wait Launch Impact
Company: Waymo
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Take-home Project
A ride-hailing or autonomous-vehicle product launches a feature called **Smart Wait**. Before a user confirms a ride, Smart Wait shows a more conservative estimated wait time, meaning the displayed estimate is usually longer than before. The goal is to improve ETA accuracy and reduce user disappointment from overly optimistic wait estimates.
After launch, the team observes:
- Quote-to-ride conversion rate decreased by 5% relative.
- Actual time to pickup, abbreviated TTP, decreased by 25% among completed rides.
A product manager claims: "Fleet efficiency improved because actual TTP decreased."
Answer the following:
1. Do you agree with the PM's conclusion? Why or why not?
2. What alternative hypotheses could explain the TTP decrease, especially in relation to the conversion drop?
3. What metrics and analyses would you use to decide whether Smart Wait was beneficial overall?
4. If you were designing an experiment for this feature, what would you randomize, what would be the primary metric, and what guardrails would you monitor?
Quick Answer: This question evaluates a data scientist's competencies in causal inference and experiment analysis, including interpreting A/B test outcomes, trade-offs between conversion and operational metrics, selection of primary and guardrail metrics, and generation of alternative hypotheses.