Diagnosing a YouTube Recommendation Engagement Drop: A Metrics Plan

You work as a product manager on YouTube and notice that users are engaging less with recommended videos than before. Which metrics would you track to understand the decline and to decide your next steps? ```hint Break the metric apart first Engagement with a recommendation happens in steps: the video has to be shown, chosen, watched and then reacted to, by some group of users on some surface. Decompose the drop along those steps and groups so that each metric you pick tells you where the change happened. ``` ### Clarifying Questions - What counts as engagement here: clicks on recommendations, watch time from them, or interactions such as likes, comments and shares? - Which recommendation surfaces are affected: the home feed, the videos suggested next to or after a video, Shorts, or all of them? - How large is the drop, did it appear suddenly or gradually, and is it measured as a total or per user? - Is overall YouTube usage also down, or only engagement that starts from a recommendation? ### What a Strong Answer Covers - A precise definition of the declining metric and a check that the drop is real rather than a logging or definition change - A decomposition of recommendation engagement into funnel steps (reach, impressions, clicks, watching, interactions) with a metric for each - Segment cuts that localize the drop: platform and app version, country, surface, user tenure, content type and format - Internal and external hypotheses (ranking or UI launches, ad load, content supply, seasonality, competing products) and the metric that confirms or rules out each - An order of investigation that maps each finding to a next step, with satisfaction and creator-side guardrails ### Follow-up Questions - Suppose the drop is limited to one country and began the week a new ranking model launched there. What do you do next? - Click-through on recommendations is steady, but likes and comments per view are falling. What could explain that, and is it a problem? - How would you tell whether users are unhappy with recommendations or simply moving their time to another YouTube surface such as Shorts or search?