Diagnosing a Falling Core Metric and Choosing What to Build to Recover It

One of your product's key metrics has started to decline. How would you work out why, and how would you then decide what to build in response? ### Clarifying Questions - Which metric is falling: an outcome for end users, such as the share of conversations an AI agent resolves without a human, a satisfaction score, or a business metric such as usage or revenue? - Is the decline a sudden step on a particular day or a gradual slide over weeks, and how large is it? - Is it visible across every customer, or concentrated in some accounts, channels or regions? - Has anything shipped or changed recently, on our side or on the customers' side? ### Part 1 — Investigate the decline What would you look at first, and in what order, to find out why the metric is falling? ```hint Rule out the dull explanations Before looking for a product cause, ask whether the number can be trusted and whether the fall comes from a change in who is being measured rather than in how they behave. ``` #### What This Part Should Cover - Checking that the drop is real (instrumentation, metric definition, data pipeline) and large relative to normal variation - Lining up internal and external changes against the timing of the decline - Decomposing and segmenting the metric to localize the drop, including a check for mix shift - Pairing quantitative cuts with qualitative evidence to confirm a root cause ### Part 2 — Decide what to build With likely causes in hand, how would you choose and sequence what the team builds to recover the metric? ```hint Value a fix by what it can recover A fix can only win back as much of the decline as its segment accounts for. Think about how you would estimate that for each option before comparing effort. ``` #### What This Part Should Cover - Separating immediate fixes such as a rollback from longer product investments - A prioritization method that links each option to the share of the decline it can recover, its confidence and its effort - How each change is validated, with guardrails that stop the metric from being recovered in a misleading way ### What a Strong Answer Covers - An investigation order that moves from data validity to timing to segmentation to a confirmed cause - A direct link between the diagnosis and the build decisions, not a generic feature list - Prioritization with stated assumptions and simple arithmetic - Guardrail metrics and a plan to confirm the recovery and catch the next decline sooner ### Follow-up Questions - The cause turns out to be one large customer whose own users changed behavior after that customer launched something new. Is this yours to fix, and what would you do? - No single cause appears: the metric is sliding slowly in every segment. How do you proceed? - Leadership wants a recovery date while the cause is still uncertain. What do you tell them?