Disney+ Metric Drop: Diagnosing a Sudden 10% Fall in Watch Time on a Recommendation Row

You are the product manager who owns the "Recommended for you" row in the Disney+ app, the personalized strip of titles on the home screen. You notice that total average minutes viewed has suddenly fallen by 10%. How would you respond? Explain how you would investigate the drop, in what order, and what you would do once you found the cause. ```hint Confirm, then locate Before forming theories, make sure the drop is real and work out whether it belongs to the row or to the whole app. Then split the row's minutes into a few factors and find which one moved. ``` ### Clarifying Questions - Is the metric minutes watched from plays that started on this row, or total minutes in the app, and is it averaged per subscriber, per active user or per session? - What is the comparison: day over day, week over week, or the same period last year, and when exactly did the drop begin? - Did the drop appear everywhere at once, or only on some devices, app versions or countries? - Were any app releases, recommendation changes, experiments or catalog changes shipped around the time it began? ### What a Strong Answer Covers - Verifying that the drop is real: tracking, attribution and dashboard changes, and whether the numerator or the denominator of the average moved - Scoping the drop to the row or the whole app, including whether viewing simply moved to other parts of the home screen - A decomposition of the row's minutes into how often it is seen, how often it leads to a play and how long each play lasts, with cuts by device, app version, country, plan and profile type - Internal and external causes checked in a sensible order, followed by a fix, a rollback decision and monitoring that catches a repeat - A judgment of impact: whether the drop threatens retention or is a harmless shift in where people find content ### Follow-up Questions - The drop is confined to the TV apps and began right after a home-screen redesign. What do you do next, and whom do you involve? - Minutes from the row fell, but total viewing in the app is flat. Is this a problem? - How would you set up alerting so that a drop like this is caught and explained within a day?