Facebook Metric Design: What Counts as a Meaningful Interaction and How to Track It

Quick Overview

A Facebook product metrics question: define a meaningful social interaction precisely enough to compute from product data, then choose the metrics to evaluate it. It tests metric design, picking a north star with supporting and guardrail metrics, validating a behavioral proxy, and spotting ways engagement metrics can be gamed.

Facebook Metric Design: What Counts as a Meaningful Interaction and How to Track It

Company: Meta

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

You are a product manager working on Facebook, the core social app. Explain how you would define a "meaningful social interaction" on the platform, concretely enough that the team could compute it from product data, and which metrics you would use to evaluate whether Facebook is delivering it. ```hint Not every interaction counts Before naming any metric, sort a few example interactions into "meaningful" and "not meaningful", such as a thoughtful reply from a cousin versus a tap on a brand's post. Then ask what separates them and which logged signals could capture that difference. ``` ### Clarifying Questions - Should the scope be only the main Facebook app (Feed, Groups, and messaging that starts from it), or the whole family of apps including Instagram and WhatsApp? - Will this definition be used to steer Feed ranking, to serve as a company goal metric, or both? A metric that drives ranking needs to be much harder to game. - Should interactions with Pages, public figures and Groups of strangers count, or only interactions between people who already know each other? - Is success judged on short-term engagement, or on longer-term outcomes such as retention and how people feel about their time on the app? ### What a Strong Answer Covers - A definition precise enough to compute: who is on each side of the interaction, which actions qualify, and what effort or reciprocity threshold separates meaningful from passive engagement, tied to why it matters for Facebook's users. - A metric hierarchy: one north-star measure, supporting metrics for content creation, consumption and responses, and the reasoning behind the choice. - A way to check the behavioral proxy against what people themselves say is meaningful. - Guardrails for gaming, divisive or low-quality engagement, and side effects on other parts of the business. - How the metrics would be segmented and used to judge experiments. ### Follow-up Questions - After a ranking change, comments per user go up, but survey respondents say their Feed feels more negative. How do you reconcile those signals? - How would your definition treat a long comment argument between two strangers in a large public Group? - A user with only a handful of friends will score low on your metric no matter what. How do you avoid building a product that ignores them?

Overview: A Facebook product metrics question: define a meaningful social interaction precisely enough to compute from product data, then choose the metrics to evaluate it. It tests metric design, picking a north star with supporting and guardrail metrics, validating a behavioral proxy, and spotting ways engagement metrics can be gamed.

Meta logo
Meta
Jan 9, 2025
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

You are a product manager working on Facebook, the core social app. Explain how you would define a "meaningful social interaction" on the platform, concretely enough that the team could compute it from product data, and which metrics you would use to evaluate whether Facebook is delivering it.

Clarifying Questions Guidance

  • Should the scope be only the main Facebook app (Feed, Groups, and messaging that starts from it), or the whole family of apps including Instagram and WhatsApp?
  • Will this definition be used to steer Feed ranking, to serve as a company goal metric, or both? A metric that drives ranking needs to be much harder to game.
  • Should interactions with Pages, public figures and Groups of strangers count, or only interactions between people who already know each other?
  • Is success judged on short-term engagement, or on longer-term outcomes such as retention and how people feel about their time on the app?

What a Strong Answer Covers Guidance

  • A definition precise enough to compute: who is on each side of the interaction, which actions qualify, and what effort or reciprocity threshold separates meaningful from passive engagement, tied to why it matters for Facebook's users.
  • A metric hierarchy: one north-star measure, supporting metrics for content creation, consumption and responses, and the reasoning behind the choice.
  • A way to check the behavioral proxy against what people themselves say is meaningful.
  • Guardrails for gaming, divisive or low-quality engagement, and side effects on other parts of the business.
  • How the metrics would be segmented and used to judge experiments.

Follow-up Questions Guidance

  • After a ranking change, comments per user go up, but survey respondents say their Feed feels more negative. How do you reconcile those signals?
  • How would your definition treat a long comment argument between two strangers in a large public Group?
  • A user with only a handful of friends will score low on your metric no matter what. How do you avoid building a product that ignores them?