Evaluate Auto-Play Impact with Key Metrics and Experiment Design
Company: Uber
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
Streaming platform considering auto-playing the next episode to improve engagement.
##### Question
What primary metrics and guardrail metrics would you track to evaluate auto-play? Design an experiment to measure the feature's impact; outline unit of randomization and duration. If churn increases in power users but total watch time rises overall, how would you decide whether to launch?
##### Hints
Discuss trade-offs, segmentation, north-star metric.
Quick Answer: This question evaluates a data scientist's competency in experiment design, metric selection, causal inference, segmentation, and trade-off analysis for product changes such as auto-play, including defining north-star and guardrail metrics.