Design a system to detect weapon posts
Company: Meta
Role: Machine Learning Engineer
Category: System Design
Difficulty: medium
Interview Round: Onsite
## System Design: Detect weapon-related posts
Design a system for a social media platform to **detect and moderate posts that contain weapons**.
A "post" may include:
- **Text** (caption/body)
- **Images** (one or more)
- Optionally **short video** (assume you can treat it as sampled frames)
### Requirements
1. **Real-time decisioning** for newly created posts:
- Either **allow**, **block**, or **send to human review**.
2. Support **high throughput** (assume millions of posts/day) with **low latency** for the user-facing publish flow.
3. Handle **model uncertainty** and reduce harm:
- Minimize false negatives for clearly dangerous content.
- Keep false positives low enough to avoid mass user frustration.
4. Provide **auditability** (why was content blocked) and **appeals**.
5. Include an approach for **continuous improvement** (feedback loop, retraining, drift detection).
### What to cover
- API + data flow and key components
- Storage and messaging choices
- ML model serving (online vs offline, multi-modal)
- Human-in-the-loop moderation workflow
- Metrics/SLOs and monitoring
- Failure modes and abuse/attack considerations (adversarial content, evasion)
Quick Answer: This question evaluates system design and machine learning engineering competencies, including multi-modal content detection, real-time model serving, scalable distributed architectures, human-in-the-loop moderation, auditability, monitoring, and adversarial robustness.