Facebook Friend Discovery: Proposing and Designing A/B Experiments

Quick Overview

A product experimentation question on which A/B tests would make it easier for people to find and add friends on Facebook. It tests mapping the friend-finding paths, writing testable hypotheses, prioritizing ideas, and designing one experiment with a sound randomization unit, success metric and guardrails on a two-sided social network.

Facebook Friend Discovery: Proposing and Designing A/B Experiments

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

On Facebook, two people become friends when one sends a friend request and the other accepts it. The product team wants to make it easier for people to find others they would like to add as friends. What A/B tests would you propose? Decide which part of the friend-finding experience to focus on, suggest a few experiments with clear hypotheses, pick the one you would run first, and explain how you would set it up and decide whether it worked. ```hint Locate the friction People reach a new friend in more than one way. Pick a path, walk through it step by step, and find where someone who wants to connect gets stuck; each test should target one of those steps. ``` ### Clarifying Questions - Does "find friends" mean locating a specific person the user already has in mind, for example through search, or discovering people they know through suggestions? - Is the aim more friend requests, more accepted connections, or connections that people go on to interact with? - Should the tests target a particular group, such as new accounts with few friends or long-time users who rarely add anyone? - Are both the mobile app and desktop in scope? ### What a Strong Answer Covers - A stated goal and a chosen friend-finding path, with the specific friction each idea addresses. - Several distinct test ideas, each with a falsifiable hypothesis, prioritized by expected impact, confidence and cost. - A complete design for the first test: randomization unit, target population, variants, duration, and how to handle the fact that a friend request involves two users. - A primary metric that reflects successful connections, with secondary metrics and guardrails for unwanted requests, privacy and overall engagement. - A decision rule for shipping, including what to do with mixed results. ### Follow-up Questions - A variant raises the number of friend requests sent but lowers the acceptance rate. Would you ship it? - If treated users send more requests, users in the control group receive more of them. How does this bias the result, and how would you design around it? - Showing why someone is suggested lifts friend adds but also raises privacy complaints. How do you decide?

Overview: A product experimentation question on which A/B tests would make it easier for people to find and add friends on Facebook. It tests mapping the friend-finding paths, writing testable hypotheses, prioritizing ideas, and designing one experiment with a sound randomization unit, success metric and guardrails on a two-sided social network.

Meta logo
Meta
Jul 15, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

On Facebook, two people become friends when one sends a friend request and the other accepts it. The product team wants to make it easier for people to find others they would like to add as friends.

What A/B tests would you propose? Decide which part of the friend-finding experience to focus on, suggest a few experiments with clear hypotheses, pick the one you would run first, and explain how you would set it up and decide whether it worked.

Clarifying Questions Guidance

  • Does "find friends" mean locating a specific person the user already has in mind, for example through search, or discovering people they know through suggestions?
  • Is the aim more friend requests, more accepted connections, or connections that people go on to interact with?
  • Should the tests target a particular group, such as new accounts with few friends or long-time users who rarely add anyone?
  • Are both the mobile app and desktop in scope?

What a Strong Answer Covers Guidance

  • A stated goal and a chosen friend-finding path, with the specific friction each idea addresses.
  • Several distinct test ideas, each with a falsifiable hypothesis, prioritized by expected impact, confidence and cost.
  • A complete design for the first test: randomization unit, target population, variants, duration, and how to handle the fact that a friend request involves two users.
  • A primary metric that reflects successful connections, with secondary metrics and guardrails for unwanted requests, privacy and overall engagement.
  • A decision rule for shipping, including what to do with mixed results.

Follow-up Questions Guidance

  • A variant raises the number of friend requests sent but lowers the acceptance rate. Would you ship it?
  • If treated users send more requests, users in the control group receive more of them. How does this bias the result, and how would you design around it?
  • Showing why someone is suggested lifts friend adds but also raises privacy complaints. How do you decide?