Choosing a North-Star Metric and Guardrails for Curated Account Lists on X

Quick Overview

A metrics question about Lists on X (formerly Twitter), which let people build custom timelines from chosen accounts without following them. It tests defining what success means for the feature, choosing a north-star metric with supporting metrics, and setting guardrails for effects on the home timeline, following and abuse.

Choosing a North-Star Metric and Guardrails for Curated Account Lists on X

Company: X (Twitter)

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

On X (formerly Twitter), Lists let a person put together a custom timeline that contains posts only from accounts they choose to add. Adding an account to a List does not require following it, so the person's main home timeline stays free of clutter. How would you measure whether Lists is a success? ```hint Follow a List through its life A person creates a List, adds accounts to it, and then reads it often, rarely or never. Ask which of these steps shows that the List is actually useful to them. ``` ### Clarifying Questions - What should Lists achieve for the company: deeper engagement from heavy users, a better experience for people who do not want to follow many accounts, or discovery of accounts? - Do Lists a person subscribes to, made by someone else, count the same as Lists they built themselves? - Are we assessing Lists as a mature feature, or measuring a recent change to it? - Should Lists be judged only on its own usage, or also on its effect on the home timeline and on following? ### What a Strong Answer Covers - The purpose of Lists and the user needs it serves, tied to company goals - A user journey from discovering Lists through creating and curating one to reading it regularly, with a metric at each step - One north-star metric that reflects the value people get, with the reasoning behind the choice - Guardrails for effects on the home timeline, on following, on the accounts that get listed, and on abuse - How to segment the data and separate the effect of Lists from the fact that heavy users are the ones who adopt it ### Follow-up Questions - After a redesign, List creation jumps but time spent reading Lists stays flat. How do you interpret that? - Heavy List users follow fewer accounts than similar users who do not use Lists. Is that a problem for the platform? - How would you show that Lists makes people more likely to stay, rather than simply being used by people who were going to stay anyway?

Overview: A metrics question about Lists on X (formerly Twitter), which let people build custom timelines from chosen accounts without following them. It tests defining what success means for the feature, choosing a north-star metric with supporting metrics, and setting guardrails for effects on the home timeline, following and abuse.

X (Twitter) logo
X (Twitter)
Sep 4, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

On X (formerly Twitter), Lists let a person put together a custom timeline that contains posts only from accounts they choose to add. Adding an account to a List does not require following it, so the person's main home timeline stays free of clutter. How would you measure whether Lists is a success?

Clarifying Questions Guidance

  • What should Lists achieve for the company: deeper engagement from heavy users, a better experience for people who do not want to follow many accounts, or discovery of accounts?
  • Do Lists a person subscribes to, made by someone else, count the same as Lists they built themselves?
  • Are we assessing Lists as a mature feature, or measuring a recent change to it?
  • Should Lists be judged only on its own usage, or also on its effect on the home timeline and on following?

What a Strong Answer Covers Guidance

  • The purpose of Lists and the user needs it serves, tied to company goals
  • A user journey from discovering Lists through creating and curating one to reading it regularly, with a metric at each step
  • One north-star metric that reflects the value people get, with the reasoning behind the choice
  • Guardrails for effects on the home timeline, on following, on the accounts that get listed, and on abuse
  • How to segment the data and separate the effect of Lists from the fact that heavy users are the ones who adopt it

Follow-up Questions Guidance

  • After a redesign, List creation jumps but time spent reading Lists stays flat. How do you interpret that?
  • Heavy List users follow fewer accounts than similar users who do not use Lists. Is that a problem for the platform?
  • How would you show that Lists makes people more likely to stay, rather than simply being used by people who were going to stay anyway?