North-star and guardrail metrics for generative AI across Meta's apps

Meta has added generative AI to many of its products: an assistant that people can chat with inside its apps, tools that create or edit images and video, and features that help businesses produce ads. As a product manager at Meta, explain how you would measure whether these AI features are succeeding. ```hint Usage is not the same as value AI entry points sit in places people already tap, so raw usage can climb without anyone being helped. Look for signals that show a person got something useful and chose to come back. ``` ### Clarifying Questions - Should the answer cover the whole set of AI features, or focus on one, such as the assistant, the image and video creation tools, or the tools for advertisers? - Whose success matters most here: people using Meta's apps, creators and businesses, or Meta's own engagement and ad revenue? - Is this for an ongoing health dashboard, or to decide whether a particular AI feature keeps getting investment? ### What a Strong Answer Covers - A goal for each kind of AI feature, and a justified choice of where to focus - A north-star metric that reflects repeated, voluntary value rather than exposure or accidental use - Supporting metrics for adoption, task success and quality, retention, and contribution to Meta's core engagement or revenue - Guardrails for safety, accuracy, trust, cannibalization of other activity, and the cost of serving AI responses - A method for establishing causality, such as long-running holdouts or experiments, instead of before-and-after comparisons ### Follow-up Questions - Daily assistant users are rising, but most sessions are a single question typed into the search bar. Is that success, and how would you tell? - How would you measure whether AI-generated images make the feed better or worse for viewers who never use the creation tools? - As usage grows, the cost of serving responses climbs sharply. How would you decide whether a feature still pays for itself?