Design measurement to detect fake accounts
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Onsite
## Context
You work on a social platform. The only product surface you can rely on is **friend requests** (sending/receiving/accepting/declining). Assume you have **no existing anti-fake model, no rules, and no established metrics**.
## Task
1. **Define “fake account” operationally.**
- What behaviors qualify (spam, scam, bot, account farming)?
- How will you handle ambiguous/gray accounts?
2. **Design the data and instrumentation.**
- What events and fields would you log for friend requests and subsequent user actions?
- What joins/identifiers are needed to track outcomes over time?
3. **Propose an initial detection approach without a model.**
- What heuristic signals or risk scoring would you start with (rate limits, graph patterns, acceptance ratios, burstiness, messaging-after-accept if available, etc.)?
- How would you choose thresholds and prevent hurting legitimate users?
4. **Measurement & evaluation plan.**
- How will you obtain labels (manual review, user reports, enforcement actions) and deal with delayed/biased labels?
- What are the **primary**, **diagnostic**, and **guardrail** metrics?
5. **Platform-level reporting.**
- How would you estimate and report the platform’s fake-account problem over time (prevalence/incidence), given that you only observe partial ground truth?
- What would you show to an executive audience vs an operational team?
Overview: This question evaluates a data scientist's competencies in fraud detection measurement, event instrumentation, labeling strategy, metric design and reporting using constrained product signals.
Read the full Meta Data Scientist interview experience this question came from
Community answers
Answer by SS
Q3: Detection Heuristics (No Model)
Risk Score = weighted combination of:
Volume score (0.4 weight): Requests sent vs. P95 over 7 days
Acceptance score (0.4 weight): Acceptance rate vs. P25 over 7 days
Account age score (0.2 weight): <7 days = high risk
Thresholds:
≥0.9 → Auto-ban
0.8-0.9 → Manual review
0.7-0.8 → Rate limit
<0.7 → No action
Protect real users: 7-day observation window before flagging
Q4: Measurement & Labels
Get labels from:
User reports ("Report as spam")
Manual review queue
Post-enforcement (banned for violations later)
Metrics:
Primary:
Precision = flagged accounts that are truly fake
Recall = true fakes that we caught
Diagnostic:
Acceptance rate (overall platform)
Time-to-detection (days until flag)
Manual review queue size
Guardrails:
False positive rate <1% (real users flagged)
DAU/WAU stable
User retention unchanged
Q5: Platform Reporting
To Executives:
"We detect X% fake accounts daily affecting Y million users"
"Estimated true rate: X-Z% (confidence interval)"
"Cost: $W in support + churn"
Show trend: Is it getting better/worse?
To Ops Teams:
Precision/recall by threshold
Detection latency distribution
False positive examples
Manual review backlog
Key point: You only see detected fakes, not all fakes → use sampling to estimate true prevalence
Bonus: One More Data Source?
Messaging volume - correlates with friend request spam, strengthens detection
Remember:
✅ Stay focused on constraints (friend requests only)
✅ Use standard metrics (precision/recall