Root-causing a 20% decline in new Facebook Group creation

Quick Overview

A metric-drop question in which the number of new Facebook Groups being created has fallen 20 percent. It tests clarifying the metric and baseline, ruling out data problems, breaking down the creation funnel, segmenting users, and ordering internal and external hypotheses before deciding what to do.

Root-causing a 20% decline in new Facebook Group creation

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

You are the product manager for Facebook Groups. The number of new groups being created has dropped by 20%. Walk through what you would do. ```hint Real, then where Before naming causes, confirm the drop is genuine and find which users, surfaces or steps of the creation flow account for it. ``` ### Clarifying Questions - Over what window did the 20% drop happen, against what baseline (the prior week, the prior quarter, the same period last year), and was it sudden or gradual? - Is the metric the number of groups created or the number of people creating groups, and is it counted when someone taps Create or when a group is fully set up? - Is the decline spread evenly, or concentrated in one platform (app versus desktop web), region or type of group? - Have related metrics moved too, such as sign-ups, daily active users or activity inside existing groups? ### What a Strong Answer Covers - Pinning down the metric's definition, window and baseline before forming hypotheses. - Ruling out logging, pipeline and definition changes first. - Breaking the metric into parts (people who could create, people who start, completion rate, groups per creator) and segmenting by platform, region, user tenure and group type. - Internal and external causes, ordered by likelihood and cost to check, each paired with the data that would confirm or rule it out. - Judging whether the drop is actually harmful (for example, fewer spam groups) and the action that follows once the cause is found. ### Follow-up Questions - The entire drop is in groups that never gain a second member. Does that change how worried you are and what you do next? - An experiment that rearranged navigation items explains half the drop, but it improves another team's key metric. How do you resolve it? - How would you monitor group creation so that a drop like this is caught within days?

Overview: A metric-drop question in which the number of new Facebook Groups being created has fallen 20 percent. It tests clarifying the metric and baseline, ruling out data problems, breaking down the creation funnel, segmenting users, and ordering internal and external hypotheses before deciding what to do.

Meta logo
Meta
Nov 22, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

You are the product manager for Facebook Groups. The number of new groups being created has dropped by 20%. Walk through what you would do.

Clarifying Questions Guidance

  • Over what window did the 20% drop happen, against what baseline (the prior week, the prior quarter, the same period last year), and was it sudden or gradual?
  • Is the metric the number of groups created or the number of people creating groups, and is it counted when someone taps Create or when a group is fully set up?
  • Is the decline spread evenly, or concentrated in one platform (app versus desktop web), region or type of group?
  • Have related metrics moved too, such as sign-ups, daily active users or activity inside existing groups?

What a Strong Answer Covers Guidance

  • Pinning down the metric's definition, window and baseline before forming hypotheses.
  • Ruling out logging, pipeline and definition changes first.
  • Breaking the metric into parts (people who could create, people who start, completion rate, groups per creator) and segmenting by platform, region, user tenure and group type.
  • Internal and external causes, ordered by likelihood and cost to check, each paired with the data that would confirm or rule it out.
  • Judging whether the drop is actually harmful (for example, fewer spam groups) and the action that follows once the cause is found.

Follow-up Questions Guidance

  • The entire drop is in groups that never gain a second member. Does that change how worried you are and what you do next?
  • An experiment that rearranged navigation items explains half the drop, but it improves another team's key metric. How do you resolve it?
  • How would you monitor group creation so that a drop like this is caught within days?