Scenario
A core product metric (checkout-completion rate) shows an unexpected change. Leadership needs a root-cause analysis and action plan within hours.
Question
Yesterday, checkout-completion rate fell by 7% while overall traffic volume stayed stable.
Assumptions to clarify (state your approach for both if needed):
-
Drop definition: 7% relative vs. 7 percentage points absolute. If unspecified, assume a 7% relative drop (e.g., from 60.0% to 55.8%).
-
"Traffic stayed stable" refers to total sessions; channel/device mix may still have shifted.
How would you:
-
Diagnose the cause within a few hours (segmentation, funnel localization, cohort/time comparisons, A/B flag changes, external events)?
-
Quantify impact on users and revenue.
-
Decide and prioritize fixes, including any immediate mitigations and follow-up experiments.
Include: what analyses to run, what data/telemetry to check, how to isolate likely causes, and how to communicate next steps.