Evaluate fake accounts and ad creation
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
Answer both of the following analytics questions.
1. **Fake accounts on a social platform**
The platform wants to reduce fake or inauthentic accounts, such as spam bots, mass-created accounts, coordinated abuse accounts, or account farms. How would you define the problem, estimate prevalence, create ongoing health metrics, and evaluate whether a detection or enforcement system is actually helping? Discuss low base rates, label quality, precision-recall tradeoffs, and how you would avoid confusing changes in detection volume with changes in true prevalence.
2. **AI-assisted ad creation for advertisers**
The ads platform is launching a feature that helps advertisers generate ad creatives using AI. How would you evaluate whether this feature should launch broadly? What primary metric would you choose, what supporting and guardrail metrics would you track, and how would you design the test? Go beyond the creative generation flow itself and consider downstream effects on advertiser outcomes, user experience, and the ads marketplace. Address selection bias, interference from the auction, and heterogeneous treatment effects across advertiser segments.
Quick Answer: This question evaluates a data scientist's competencies in measurement and experimentation, covering prevalence estimation and detection system evaluation for fake accounts, metrics design and label quality assessment, precision–recall tradeoffs, and causal experimentation and marketplace impact analysis for AI-assisted ad creation.