Facebook Ads Team Metrics and a Launch Test for AI-Generated Ads for First-Time Sellers

You are the product manager for Facebook Ads. First define how the team's success should be measured, then decide whether to launch generative AI features in the ad-creation flow for new sellers. ### Clarifying Questions - Does "the team" own the whole ads business, or mainly the tools advertisers use to create and run ads? - Who counts as a new seller: a business that has never run an ad on Facebook, or a merchant who is new to selling on Meta's apps altogether? - Which generative capabilities are in scope: ad copy and headlines, images or backgrounds built from a product photo, or a complete ad the seller only approves? - Do generated ads go through the same ad review and policy checks as ads built by hand? ### Part 1 — Success metrics for the Facebook Ads team How would you define success metrics for the team? ```hint Three parties Advertisers, the people who see ads, and Meta each win in different ways. Look for what ad revenue alone could hide, and decide whose harm should cap the others' gains. ``` #### What This Part Should Cover - A goal for the team and the parties it serves - A north-star metric and why it is better than revenue on its own - Supporting metrics along the advertiser journey, plus guardrails for the people who see ads and for ad quality ### Part 2 — Deciding whether to ship GenAI ad creation for new sellers The team is weighing adding generative AI to the ad-creation process for sellers who are new to advertising. How would you decide whether to launch it? ```hint Write the no-ship result first More ads created is not the same as more sellers who succeed with ads. Decide which result would stop the launch before you design the test. ``` #### What This Part Should Cover - The new-seller problem the feature solves and the hypothesis it tests - An experiment with a randomization unit, primary metric, guardrails and a duration that can show retention - Ship, iterate and do-not-ship criteria, including accuracy, policy and trust risks of generated content ### What a Strong Answer Covers - Part 2 decision metrics that trace back to the Part 1 framework rather than a separate, ad hoc list - Both short-term effects (first ads published) and long-term ones (ad performance, sellers who keep advertising) - Risks specific to generated ads: invented product claims, policy violations, and how people react to synthetic ads - Cost of generation weighed against the incremental value of the launch ### Follow-up Questions - The test shows more new sellers publish a first ad, but their retention after two months is unchanged. Do you ship? - Treatment and control ads compete in the same auctions. How does that bias your result, and how would you correct for it? - How would you find out whether people trust generated ads less, and what would you do if they do?