Measure fake account prevalence
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
A social platform is concerned about fake accounts. Leadership wants to understand how serious the problem is and whether a new detection model or enforcement policy should be launched.
How would you approach this problem?
In your answer, discuss:
- how you would define a fake account operationally
- how you would estimate prevalence and business impact
- what metrics you would use to evaluate a detection system
- how you would deal with class imbalance, delayed labels, and selection bias from user reports or manual reviews
- how you would measure false-positive harm to legitimate users
- how you would evaluate a new intervention online, given that attackers may adapt over time
Quick Answer: This question evaluates a data scientist's competency in fraud measurement, statistical estimation, experimental design, and model evaluation for detection systems, including consideration of class imbalance, delayed labels, selection bias, false-positive harm, and adaptive attacker behavior.