Design a weapon-ad harmful content detection system
Company: Meta
Role: Machine Learning Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
## Scenario
You are building a system for an ads platform that must **detect advertisements that contain weapons** (e.g., guns, knives) and prevent policy-violating ads from being served.
Ads may include:
- Images (single/multi-image)
- Video
- Text (title/description)
- Landing page URL/content
## What to design
Design an end-to-end **harmful content detection** system that:
1. Screens ads at submission time (and/or pre-serve) with low latency.
2. Supports high throughput (large ad volume).
3. Provides a decision: **approve / reject / send to human review**.
4. Handles ambiguous cases, adversarial attempts, and policy changes.
5. Enables continuous improvement via feedback and retraining.
## Deliverables
In your design, cover:
- Functional + non-functional requirements (latency, throughput, availability)
- High-level architecture and key components
- Model approach (multi-modal), feature extraction, inference serving
- Data labeling, human review workflow, and feedback loops
- Monitoring/metrics and quality evaluation (precision/recall, false positives)
- Safety, privacy, and abuse considerations
Quick Answer: This question evaluates skills in end-to-end system design and applied machine learning for multi-modal harmful content detection, covering scalability, low-latency inference, human-in-the-loop workflows, feedback-driven retraining, and safety/privacy considerations.