Choosing Primary, Funnel and Guardrail Metrics to Judge a New Product Launch

You are the product manager for a product that is about to launch. Explain which metrics you would track to decide whether the launch succeeds or fails. The interviewer does not name the product, so pick one (or agree on one with the interviewer), then explain what you would measure, why each metric matters, and how you would turn the numbers into a clear success-or-failure call. ```hint Anchor on the value moment Before listing metrics, name the moment when a user first gets what the product promises, and the business reason for launching it. A metric that traces back to neither is probably a vanity number. ``` ### Clarifying Questions - Is this a brand-new standalone product, or a feature launching inside an app that already has a large user base? - What decision rides on the result: expanding a limited test to everyone, continuing to fund the team, or shutting the product down? - Over what time window will success be judged: the first few weeks, or a quarter or more after launch? - Is a staged rollout or a holdout group possible, or will every user get the product at once? ### What a Strong Answer Covers - A named product, its target users and the business reason for launching it, stated before any metric. - One primary success metric tied to the moment users get value, with reasons it beats obvious alternatives such as downloads or daily active users. - Supporting metrics that follow users through adoption, activation, engagement and retention, covering every side of the product if it has more than one. - Guardrail metrics that catch harm to quality, trust and the company's existing products. - Success criteria: targets set before launch, a comparison against a control or baseline, the time horizon for each read, and how each outcome maps to a decision. ### Follow-up Questions - Your primary metric beats its target, but four-week retention of new users keeps falling. Do you call the launch a success? - Network effects make a user-level A/B test unreliable for this product. How would you measure its incremental impact? - Leadership wants one number on the launch dashboard. Which would you pick, and what would that number hide?