North-Star and Guardrail Metrics for Facebook's Notification System

Facebook alerts people when something involving them or their friends happens: a comment or reaction on their post, a tag in someone else's photo, a new friend request, a friend starting a live video, fresh activity in a group they belong to. These alerts arrive through the bell icon inside the app, as mobile push notifications, and by email. You own Facebook notifications as a product manager. How would you measure whether notifications are working? Name the metric you would treat as primary, the metrics that support it, and the signals that would tell you notifications are hurting the experience. ```hint Look past the tap A tap is only the start. Consider what a person does in the minutes after opening a notification, and what a person who receives too many of them eventually does to their settings. ``` ### Clarifying Questions - Should the metrics cover every delivery channel (bell, mobile push, email), or focus on one channel such as push? - Are we measuring the notification system as a whole, or one notification type such as tags or comment replies? - Is the goal ongoing health monitoring, or judging a specific change, such as a new model that decides which notifications to send and when? - Are Messenger chat alerts in scope, or only notifications from the main Facebook app? ### What a Strong Answer Covers - A clear statement of why notifications exist for Facebook, for the recipient, and for the person whose action triggered the alert, and what a valuable notification achieves. - A funnel from eligible event to sent, delivered, seen, opened and a meaningful follow-up action, with metrics placed at the right stages and split by notification type and channel. - One justified primary metric, with the reasons other candidates (click-through rate, daily active users) were not chosen. - Guardrails for fatigue and harm: push disables, per-type opt-outs, "see fewer" feedback, uninstalls and longer-term retention. - How the metrics would be read: segmentation, holdouts or A/B tests to measure incremental value, and the risk of rewarding volume over quality. ### Follow-up Questions - After a ranking change, your primary metric rises but so does the share of users disabling push. How do you decide whether to ship? - Many people would have opened Facebook that day anyway. How would you measure the incremental value notifications create? - Would you measure an alert about a friend's comment on your post the same way as an alert nudging you to share a memory? Why or why not?