Designing a Metrics Framework to Judge Whether a Product Is Working

In a product manager interview, you are asked which kinds of metrics you rely on to judge whether a product is succeeding. Describe the categories of metrics you would use, explain how they relate to one another, and show how you would narrow them down to the few that matter for one specific product. ```hint Begin with the value, not the dashboard Decide what the product does for the people who use it and how the business benefits; the metrics should follow from that, and the right set shifts as the product matures. ``` ### Clarifying Questions - Should the answer cover products in general, or be anchored on one product, for example Indeed's job search, which serves both job seekers and employers? - Is the product new and still proving that people want it, or established and focused on growth and revenue? - Are these metrics for tracking the product's ongoing health, for judging a single feature launch, or both? ### What a Strong Answer Covers - Metrics derived from the product's goal and the value it creates, rather than a generic list. - A layered set: one top-level success metric, the driver metrics along the user journey that move it, and guardrails that catch harm. - When the product serves more than one group, as a job marketplace does, measures for each side and for the quality of the match between them. - How the choice of metrics changes with the product's stage and with the decision each metric must inform. - What makes a metric useful (measurable, responsive, hard to game, leading or lagging) and how to avoid vanity numbers. ### Follow-up Questions - After a redesign, job seekers submit more applications, but employers say the applicants are less qualified. Is the product more successful? - Your top-level metric rises while the share of new users who come back falls. How do you interpret that? - The outcome you care most about, a hire, can take weeks to observe. Which metric would you use to judge a short A/B test?