Metric Diagnosis: Investigating a Sharp Fall in Weekly Usage on Android

Weekly usage of a consumer app on Android has dropped suddenly. How would you work out what happened? Describe the order in which you would investigate, the data you would look at in each step, and how you would decide which explanation is the real one. ```hint Let "sudden" narrow the search A slow decline and an overnight step point to different kinds of causes. What kinds of events change Android usage at one specific moment, and how could you tell them apart in the data? ``` ### Clarifying Questions - Which Meta app is this (Facebook, Instagram, Messenger or WhatsApp), and does "weekly usage" mean weekly active users, sessions per weekly user or time spent? - How large is the drop, when did it start, and did it happen within a day or over several days? - Did iOS and web move during the same period, or only Android? - Is the drop global, or limited to certain countries? ### What a Strong Answer Covers - Defining the metric, the size and timing of the drop, and whether other platforms moved - Confirming the data and logging are sound before treating the drop as real - Checking internal changes (app releases, server deploys, experiments, outages) and external ones (Android and app store changes, holidays, regional events, competitors) - Localizing the drop by segmenting on app version, OS version, device maker, country, user type and entry point, and decomposing the metric - A prioritized set of hypotheses, how each is confirmed, and the fix, communication and prevention ### Follow-up Questions - The drop is limited to one phone maker's devices that just received a system update. What would you check next, and what could your team do without that phone maker's help? - Weekly users fell while daily users stayed flat over the same days. What does that suggest? - The cause turns out to be a logging bug rather than lost users. What do you do about the dashboards, the experiments that ran during that window, and future alerting?