Root-causing a 45% August utilization drop for an app in Italy and planning a fix

Quick Overview

A metric-diagnosis case in which an app's utilization in Italy fell 45% in August, and the candidate must give the most likely cause and draft a plan to fix it. It tests defining the metric and baseline, ruling out data issues, structuring internal and external hypotheses, and matching the fix to the confirmed cause.

Root-causing a 45% August utilization drop for an app in Italy and planning a fix

Company: Google

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

You are the product manager for an app called XYZ. In August, its utilization in Italy fell by 45%. Give the most likely reason for the drop and draft a plan to fix it, including how you would confirm the cause before acting on it. ```hint Fix the baseline first Before listing causes, pin down what the 45% is measured against and whether the same pattern shows up anywhere else; the answer to both narrows the list of plausible reasons considerably. ``` ### Clarifying Questions - What does "utilization" measure for XYZ: daily active users, sessions per user, time spent, or completed transactions? - Is the 45% drop relative to July, or relative to August of last year? - Did the drop happen suddenly on particular days or build through the month, and is it confined to Italy? - What does XYZ do, and how much of its use in Italy is tied to work, study or a daily commute? ### What a Strong Answer Covers - A precise definition of the metric and the comparison period before any hypotheses - A check that the drop is real rather than a tracking, attribution or definition artifact - Hypotheses split into internal changes and external factors, tested in a sensible order with the data that would confirm or rule out each - A committed most-likely reason given the facts in the prompt, plus the evidence that would change it - A fix plan matched to the cause, with owners, a timeline and the metric that would show recovery ### Follow-up Questions - If last August showed a similar drop, what would you change about how the team monitors this metric? - Suppose the drop is concentrated among users on the newest app version on one platform. What do you do in the first 48 hours? - How would you tell leadership whether this needs a product fix or no action at all?

Overview: A metric-diagnosis case in which an app's utilization in Italy fell 45% in August, and the candidate must give the most likely cause and draft a plan to fix it. It tests defining the metric and baseline, ruling out data issues, structuring internal and external hypotheses, and matching the fix to the confirmed cause.

Google logo
Google
Oct 27, 2023
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

You are the product manager for an app called XYZ. In August, its utilization in Italy fell by 45%. Give the most likely reason for the drop and draft a plan to fix it, including how you would confirm the cause before acting on it.

Clarifying Questions Guidance

  • What does "utilization" measure for XYZ: daily active users, sessions per user, time spent, or completed transactions?
  • Is the 45% drop relative to July, or relative to August of last year?
  • Did the drop happen suddenly on particular days or build through the month, and is it confined to Italy?
  • What does XYZ do, and how much of its use in Italy is tied to work, study or a daily commute?

What a Strong Answer Covers Guidance

  • A precise definition of the metric and the comparison period before any hypotheses
  • A check that the drop is real rather than a tracking, attribution or definition artifact
  • Hypotheses split into internal changes and external factors, tested in a sensible order with the data that would confirm or rule out each
  • A committed most-likely reason given the facts in the prompt, plus the evidence that would change it
  • A fix plan matched to the cause, with owners, a timeline and the metric that would show recovery

Follow-up Questions Guidance

  • If last August showed a similar drop, what would you change about how the team monitors this metric?
  • Suppose the drop is concentrated among users on the newest app version on one platform. What do you do in the first 48 hours?
  • How would you tell leadership whether this needs a product fix or no action at all?