Identify Causes and Validate Web Product Performance Drop
Overview
This question evaluates root-cause analysis, data validation, causal inference, and experimentation design skills for diagnosing unexpected drops in DAU and conversion rates within the Analytics & Experimentation domain.
Amazon
Aug 4, 2025, 10:55 AM
Data Scientist
Technical Screen
Analytics & Experimentation
0
0
Scenario
Daily active users (DAU) and conversion rate for a web product unexpectedly drop.
Tasks
- Enumerate plausible root causes and describe the specific analyses you would run to validate or rule out each cause.
- Propose how you would design an A/B experiment to test a fix for the identified issue.
- Specify primary, secondary, and guardrail metrics to track in the experiment, and explain why.
Hints
- Consider instrumentation/data pipeline issues, release rollouts, and external events.
- Outline power, sample size, expected duration, and metric sensitivity/variance for the experiment.
Solution
Login required
Show