Tell Whether a New Facebook Feature Changed Overall Platform Engagement

You are the product manager for Facebook. Your team just launched a new feature, X, and it reports 20% engagement. How would you tell whether engagement across Facebook as a whole has gone up, gone down or stayed the same? ```hint Separate the feature from the platform Ask where the time and actions now going into X came from, and what you would need to observe to tell new engagement apart from engagement that moved over from other parts of Facebook. ``` ### Clarifying Questions - What does "20% engagement" measure: the share of people shown X who interacted with it, a 20% rise over some baseline, or something else, and over what period? - How should overall engagement be defined for this purpose: daily active people, time spent, interactions such as posts, comments and shares, or a combination? - Did X reach everyone at once, or was it released gradually or with a group held back? - Which apps, surfaces and regions received X? ### What a Strong Answer Covers - An interpretation of the 20% figure and why it cannot answer the question on its own - A definition of overall engagement, with one primary metric and the component metrics beneath it - A method that attributes any change to X rather than to seasonality, other launches or novelty, plus a fallback if a clean comparison is not available - A check for cannibalization and spillover across Facebook's other surfaces - A decision rule for calling the result up, down or flat, covering statistical and practical significance and guardrails ### Follow-up Questions - The comparison shows time spent rising but comments and shares falling. Do you keep X? - Content made with X is also seen by people who do not have X. How does that affect your comparison, and how would you correct for it? - Use of X fades after its first few weeks. How do you separate novelty from lasting value?