Building a Metrics Framework to Evaluate Google Hangouts

Imagine you are the product manager for Google Hangouts, Google's product for one-to-one and group text chat, voice calls and video calls. How would you measure whether Hangouts is successful? Define what success means for this product, identify the user behaviors that show it is delivering value, and recommend the small set of metrics you would watch most closely. ```hint Start from one conversation Before naming any metric, describe what a good outcome looks like for the people in a single call or chat, then walk through the steps from setting it up to ending it and note where that outcome can break down. ``` ### Clarifying Questions - Should the measurement focus on video and voice calls, on text chat, or on the product as a whole? - Is the interest in personal use between friends and family, workplace use through Google's business tools, or both? - Does "success" mean adding users, deeper engagement, better retention, or winning meetings from competing calling tools? - Which surfaces count: the standalone apps, the web, and the versions built into Gmail and Google Calendar? ### What a Strong Answer Covers - A stated purpose for Hangouts and a working definition of success, with the scope settled before any metric is named - The user roles involved, such as the person who sets up a call and the people who join it, and the journey of a call or chat used to find where value is created or lost - One primary metric, precisely defined, with an argument for why it reflects value delivered rather than raw activity - Supporting metrics along that journey, plus guardrails that would reveal a gain in the primary metric that comes with a worse experience - How the metrics are segmented and read together: which lead, which lag, and how to interpret them when they move in opposite directions ### Follow-up Questions - If the number of people making calls rose after a redesign but average call length fell, how would you decide whether the change was good? - Many workplace meetings move to a competing tool without ever touching Hangouts. How would you detect that you are losing them? - How would your metric set change if leadership's goal were attracting new users rather than engaging existing ones?