Metrics plan to judge whether Facebook's friending feature is working

Quick Overview

A product metrics exercise on judging whether Facebook's friends feature is succeeding. It tests stating the feature's purpose, building metrics across the friending journey for both requesters and recipients, choosing a primary measure with guardrails, and reasoning about network spillover when testing changes.

Metrics plan to judge whether Facebook's friending feature is working

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

People on Facebook connect as friends: they find each other, send and accept friend requests, and then see and react to each other's posts, photos and messages. As a product manager at Meta, explain how you would tell whether this friends feature is succeeding. Say what the feature is for, lay out the metrics you would track, choose the one you would treat as the primary measure of success, and explain what would stop that measure from rewarding the wrong behavior. ```hint Follow one friendship Trace a friendship from the moment one person first sees the other's name to several weeks after the request is accepted, from the point of view of both the sender and the recipient, and ask what success looks like at each step. ``` ### Clarifying Questions - Does "friends feature" cover the whole friending flow (search, suggested friends, requests and the friends list) or one piece of it, such as friend suggestions? - Is this a health check on a mature feature, or an evaluation of a recent change to it? - Should success be judged for new users building their first network, for established users, or for both? ### What a Strong Answer Covers - A stated purpose for the feature, covering what each person in a friendship gains and why it matters to Facebook - Metrics for each stage of the friending journey, for both the person sending a request and the person receiving it - One primary success metric, why it was chosen, and why simpler counts could mislead - Guardrails that catch unwanted, spammy or privacy-invasive friending - Segments to watch, and how to read changes causally when every friendship involves two users ### Follow-up Questions - After a change to friend suggestions, the acceptance rate rises but new friendships produce fewer interactions than before. What could explain it, and what would you do? - How would you A/B test a change to friend suggestions when people in the test and control groups can send requests to each other? - Your primary metric drops sharply in one week. Walk through how you would find the cause.

Overview: A product metrics exercise on judging whether Facebook's friends feature is succeeding. It tests stating the feature's purpose, building metrics across the friending journey for both requesters and recipients, choosing a primary measure with guardrails, and reasoning about network spillover when testing changes.

Meta logo
Meta
Mar 29, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

People on Facebook connect as friends: they find each other, send and accept friend requests, and then see and react to each other's posts, photos and messages. As a product manager at Meta, explain how you would tell whether this friends feature is succeeding. Say what the feature is for, lay out the metrics you would track, choose the one you would treat as the primary measure of success, and explain what would stop that measure from rewarding the wrong behavior.

Clarifying Questions Guidance

  • Does "friends feature" cover the whole friending flow (search, suggested friends, requests and the friends list) or one piece of it, such as friend suggestions?
  • Is this a health check on a mature feature, or an evaluation of a recent change to it?
  • Should success be judged for new users building their first network, for established users, or for both?

What a Strong Answer Covers Guidance

  • A stated purpose for the feature, covering what each person in a friendship gains and why it matters to Facebook
  • Metrics for each stage of the friending journey, for both the person sending a request and the person receiving it
  • One primary success metric, why it was chosen, and why simpler counts could mislead
  • Guardrails that catch unwanted, spammy or privacy-invasive friending
  • Segments to watch, and how to read changes causally when every friendship involves two users

Follow-up Questions Guidance

  • After a change to friend suggestions, the acceptance rate rises but new friendships produce fewer interactions than before. What could explain it, and what would you do?
  • How would you A/B test a change to friend suggestions when people in the test and control groups can send requests to each other?
  • Your primary metric drops sharply in one week. Walk through how you would find the cause.