Success Metrics for a Read-Receipt Feature in Facebook Messaging

Quick Overview

A product metrics question about a read-receipt feature newly launched in Facebook messaging. It asks which metrics to collect and how to judge whether the launch succeeded, testing goal setting, sender and recipient behavior, guardrails, and experiment design for a feature that involves two people.

Success Metrics for a Read-Receipt Feature in Facebook Messaging

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

Facebook has just launched read receipts for messages: once a recipient opens a message, the sender can see that it has been read. You are the product manager for the feature. Which metrics would you collect, and how would you determine whether the launch was a success? ```hint Look at both ends of the thread Think through how a sender behaves differently once they can see that a message was read, how a recipient behaves differently once they know the sender can see it, and which of those changes Facebook actually wants. ``` ### Clarifying Questions - Does the feature apply only to one-to-one chats, or also to group threads, where some members may have read a message and others not? - Was it released to everyone at once, or is there a holdout or staged rollout to compare against? - Can recipients turn read receipts off, and if they do, do they also stop seeing other people's receipts? - Is the aim more replies and conversations, or a less uncertain experience for senders even if message volume stays flat? ### What a Strong Answer Covers - The feature's purpose for senders and for recipients, tied to what Facebook wants from messaging - Metrics spanning whether receipts work reliably, how senders and recipients change their behavior, and the health of conversations overall - One primary success metric with guardrails, and why it is harder to game than the alternatives - A way to attribute changes to the feature, given that a receipt involves two people, plus a decision rule set before reading results - Possible harms to recipients, such as pressure to reply or privacy concerns, and how they would show up in the data ### Follow-up Questions - Replies got faster, but people who receive many messages started opening them later. How would you read that result? - How would your metrics and success criteria change for group chats? - Should receipts appear on message requests from people the recipient does not know, and how would you decide?

Overview: A product metrics question about a read-receipt feature newly launched in Facebook messaging. It asks which metrics to collect and how to judge whether the launch succeeded, testing goal setting, sender and recipient behavior, guardrails, and experiment design for a feature that involves two people.

Meta logo
Meta
Dec 22, 2023
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

Facebook has just launched read receipts for messages: once a recipient opens a message, the sender can see that it has been read. You are the product manager for the feature. Which metrics would you collect, and how would you determine whether the launch was a success?

Clarifying Questions Guidance

  • Does the feature apply only to one-to-one chats, or also to group threads, where some members may have read a message and others not?
  • Was it released to everyone at once, or is there a holdout or staged rollout to compare against?
  • Can recipients turn read receipts off, and if they do, do they also stop seeing other people's receipts?
  • Is the aim more replies and conversations, or a less uncertain experience for senders even if message volume stays flat?

What a Strong Answer Covers Guidance

  • The feature's purpose for senders and for recipients, tied to what Facebook wants from messaging
  • Metrics spanning whether receipts work reliably, how senders and recipients change their behavior, and the health of conversations overall
  • One primary success metric with guardrails, and why it is harder to game than the alternatives
  • A way to attribute changes to the feature, given that a receipt involves two people, plus a decision rule set before reading results
  • Possible harms to recipients, such as pressure to reply or privacy concerns, and how they would show up in the data

Follow-up Questions Guidance

  • Replies got faster, but people who receive many messages started opening them later. How would you read that result?
  • How would your metrics and success criteria change for group chats?
  • Should receipts appear on message requests from people the recipient does not know, and how would you decide?