Judging whether Instagram's removal of like totals from posts is working

Quick Overview

A product metrics question about Instagram's decision to hide like counts on posts. It tests whether a candidate can tie success metrics to the goal behind the change, weigh the effect on posters, viewers, creators and advertisers, set guardrails, and design a test that accounts for network effects and novelty.

Judging whether Instagram's removal of like totals from posts is working

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

Instagram has decided to stop showing like counts on posts. As the product manager, how would you measure whether this change is a success? Explain what it is trying to achieve, which metrics you would track, and how you would judge the result. ```hint Start from the problem Before naming metrics, work out what problem visible like counts caused and for whom; success should be defined by that problem shrinking. ``` ### Clarifying Questions - Are counts hidden only from other people, with authors still able to see the count on their own posts, or hidden from everyone? - Does the change cover ads and video posts as well as regular feed posts? - Is this a test in a few markets or a global launch, and can people choose to see counts again? - What prompted the change: pressure and comparison felt by people who post, people liking whatever is already popular, or likes being bought to look popular? ### What a Strong Answer Covers - The goal behind hiding counts, tied to a specific problem and the people it affects - The groups the change affects, including those who could lose from it - A primary success metric that reflects the goal, with supporting and guardrail metrics - A test or rollout design that isolates the change's effect and allows time for behavior to settle - How the results would be read, including the outcome that would lead you to adjust or reverse the change ### Follow-up Questions - Suppose casual users post more but the biggest creators post less and some move to other apps. Is the change a success? - How would you measure social pressure or wellbeing, which no log captures directly? - How would network effects bias a test that randomizes individual users, and what would you do about it?

Overview: A product metrics question about Instagram's decision to hide like counts on posts. It tests whether a candidate can tie success metrics to the goal behind the change, weigh the effect on posters, viewers, creators and advertisers, set guardrails, and design a test that accounts for network effects and novelty.

Meta logo
Meta
Feb 13, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

Instagram has decided to stop showing like counts on posts. As the product manager, how would you measure whether this change is a success? Explain what it is trying to achieve, which metrics you would track, and how you would judge the result.

Clarifying Questions Guidance

  • Are counts hidden only from other people, with authors still able to see the count on their own posts, or hidden from everyone?
  • Does the change cover ads and video posts as well as regular feed posts?
  • Is this a test in a few markets or a global launch, and can people choose to see counts again?
  • What prompted the change: pressure and comparison felt by people who post, people liking whatever is already popular, or likes being bought to look popular?

What a Strong Answer Covers Guidance

  • The goal behind hiding counts, tied to a specific problem and the people it affects
  • The groups the change affects, including those who could lose from it
  • A primary success metric that reflects the goal, with supporting and guardrail metrics
  • A test or rollout design that isolates the change's effect and allows time for behavior to settle
  • How the results would be read, including the outcome that would lead you to adjust or reverse the change

Follow-up Questions Guidance

  • Suppose casual users post more but the biggest creators post less and some move to other apps. Is the change a success?
  • How would you measure social pressure or wellbeing, which no log captures directly?
  • How would network effects bias a test that randomizes individual users, and what would you do about it?