Facebook reactions: test demand for one more reaction and define success metrics

Quick Overview

An analytical product question about whether Facebook should add a seventh reaction to posts and how to judge it after launch. It tests how a candidate finds evidence of an unmet need, weighs the cost of another option, and designs success metrics and an experiment for a feature that every viewer and poster sees.

Facebook reactions: test demand for one more reaction and define success metrics

Company: Coinbase

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

Facebook offers six reactions on posts and is weighing whether to add a seventh. How would you decide whether it is needed, and how would you tell whether it succeeded? ### Clarifying Questions - Has the team already proposed a specific emotion for the seventh reaction, or is finding a missing emotion part of the task? - What is the new reaction meant to achieve: letting people express something they cannot today, raising engagement, or giving feed ranking a clearer signal? - Would it appear only on feed posts, or also on comments, Messenger and the other surfaces where reactions show up? - Would it launch worldwide at once or in selected markets first? ### Part 1 — Decide whether a seventh reaction is needed What evidence would show that people have a feeling the current six reactions cannot express, and what would lead you to decide against adding one? ```hint Watch what people do instead Think about how people behave on a post when none of the six reactions fits how they feel, and which of those behaviors already show up in the data. ``` #### What This Part Should Cover - Behavioral and research evidence of an unmet expressive need, and how large it is - The costs and risks of a seventh option, such as a crowded picker, overlap with existing reactions and misuse - A clear go or no-go rule, including alternatives to a permanent new reaction ### Part 2 — Measure success after launch Assume the team builds it. How would you test it and decide whether it worked? ```hint Popular is not the same as successful Picture what usage of the new reaction could look like even if it added nothing overall, and think about who on a post is affected besides the person reacting. ``` #### What This Part Should Cover - A primary success metric tied to the goal, plus adoption and guardrail metrics - An experiment design that accounts for reactions being visible to posters and other viewers - How novelty effects and launch thresholds are handled ### What a Strong Answer Covers - Framing reactions around low-effort, meaningful interaction between people on Facebook - Consistency between the need found in Part 1 and the success criteria in Part 2 - Attention to both sides of a reaction: the person reacting and the poster receiving it - Differences by market and culture in how emotions and their icons are read ### Follow-up Questions - Adoption of the new reaction is strong, but total reactions per post did not change. Is that a success? - The new reaction is being used heavily as sarcasm or to pile on certain posts. What do you do? - Would you consider replacing the least-used existing reaction instead of adding a seventh, and how would you decide?

Overview: An analytical product question about whether Facebook should add a seventh reaction to posts and how to judge it after launch. It tests how a candidate finds evidence of an unmet need, weighs the cost of another option, and designs success metrics and an experiment for a feature that every viewer and poster sees.

Coinbase logo
Coinbase
Aug 12, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

Facebook offers six reactions on posts and is weighing whether to add a seventh. How would you decide whether it is needed, and how would you tell whether it succeeded?

Clarifying Questions Guidance

  • Has the team already proposed a specific emotion for the seventh reaction, or is finding a missing emotion part of the task?
  • What is the new reaction meant to achieve: letting people express something they cannot today, raising engagement, or giving feed ranking a clearer signal?
  • Would it appear only on feed posts, or also on comments, Messenger and the other surfaces where reactions show up?
  • Would it launch worldwide at once or in selected markets first?

Part 1 — Decide whether a seventh reaction is needed

What evidence would show that people have a feeling the current six reactions cannot express, and what would lead you to decide against adding one?

What This Part Should Cover Guidance

  • Behavioral and research evidence of an unmet expressive need, and how large it is
  • The costs and risks of a seventh option, such as a crowded picker, overlap with existing reactions and misuse
  • A clear go or no-go rule, including alternatives to a permanent new reaction

Part 2 — Measure success after launch

Assume the team builds it. How would you test it and decide whether it worked?

What This Part Should Cover Guidance

  • A primary success metric tied to the goal, plus adoption and guardrail metrics
  • An experiment design that accounts for reactions being visible to posters and other viewers
  • How novelty effects and launch thresholds are handled

What a Strong Answer Covers Guidance

  • Framing reactions around low-effort, meaningful interaction between people on Facebook
  • Consistency between the need found in Part 1 and the success criteria in Part 2
  • Attention to both sides of a reaction: the person reacting and the poster receiving it
  • Differences by market and culture in how emotions and their icons are read

Follow-up Questions Guidance

  • Adoption of the new reaction is strong, but total reactions per post did not change. Is that a success?
  • The new reaction is being used heavily as sarcasm or to pile on certain posts. What do you do?
  • Would you consider replacing the least-used existing reaction instead of adding a seventh, and how would you decide?