Testing a Friends vs Media Split of Facebook's Feed: Metrics, Blind Spots, Launch Market

Facebook is weighing a change that would split the News Feed into two separate feeds. A **Friends and Family** feed would show posts from the people a user knows, and a **Media** feed would hold the commercial side: ads, posts from businesses and other content that does not come from friends or family. The team wants to test the change before deciding. The interviewer asks three things in turn: how you would validate the split and with which metrics, what is wrong with those metrics, and which market you would launch the test in first because it is lower risk, and why. ### Clarifying Questions - How would the split appear in the app: two tabs, a toggle or two separate entry points, and which feed opens by default? - Where does a friend's share of a publisher's article or a business post go: the Friends and Family feed, because a friend shared it, or the Media feed, because of what it links to? - What prompted the idea: falling engagement with friends' posts, complaints about too much commercial content, or a business goal? - Is the change for the mobile apps only or for every platform, and is the eventual target every market? ### Part 1 — Validate the split How would you decide whether splitting the feed is the right call? Describe the metrics you would use and how you would set up the test that produces them. ```hint Who gains and who loses Before listing metrics, ask what the split is supposed to improve and which groups it changes things for, beyond the person scrolling. Each group needs something you can measure. ``` #### What This Part Should Cover - A stated hypothesis and a success definition fixed before the test runs. - A primary metric, supporting metrics and guardrails that cover viewers, the friends who post, businesses and publishers, and ads revenue. - An experiment design that suits a change touching both the social graph and the ads business. ### Part 2 — Downsides of your metrics What are the weaknesses of the metrics you chose? Explain how each one could mislead you and what you would add or change to guard against it. ```hint Make each metric lie For each metric, think of a way it could improve while users are worse off, and ask whether your test setup can measure it fairly when friends end up in different groups. ``` #### What This Part Should Cover - Metric-by-metric blind spots, such as engagement that rises mechanically or time spent that reflects friction. - Measurement problems that come from the test setup itself, not only from the metric definitions. - A concrete mitigation for each weakness. ### Part 3 — A lower-risk launch market Name the market where you would launch the test first because you judge it lower risk, and explain why it is lower risk than the alternatives. ```hint Two sides of the choice List what the company could lose in a test market and what it needs from that market to trust the result; the right choice keeps the first small without giving up the second. ``` #### What This Part Should Cover - Explicit criteria for "lower risk", covering revenue, users, publishers and public scrutiny. - What the market must still offer for the result to be trustworthy and to generalize. - A specific recommendation and its known weaknesses. ### What a Strong Answer Covers - A clear statement of what the split is meant to achieve, with a decision rule fixed before the test. - Ranked metrics that span every affected group: viewers, posting friends, Pages and publishers, and advertisers. - An understanding of the network effects and ad-auction effects that a simple user-level A/B test misses. - Honest metric weaknesses with mitigations, and a launch market argued from explicit criteria. ### Follow-up Questions - Friends and Family interactions rise, but total time on Facebook and ads revenue fall. Would you ship, and what would you need to see first? - How would you measure the impact on Pages and publishers that depend on reach from the main feed, and what would you do if it is severe? - If the test succeeds in your launch market, what would make you doubt that the result will hold in Facebook's largest markets?