Design a weapon-ad harmful content detection system
Quick Overview
This question evaluates skills in end-to-end system design and applied machine learning for multi-modal harmful content detection, covering scalability, low-latency inference, human-in-the-loop workflows, feedback-driven retraining, and safety/privacy considerations.
Meta
Jan 22, 2026, 12:00 AM
Machine Learning Engineer
Technical Screen
System Design
7
0
Loading...
Scenario
You are building a system for an ads platform that must detect advertisements that contain weapons (e.g., guns, knives) and prevent policy-violating ads from being served.
Ads may include:
- Images (single/multi-image)
- Video
- Text (title/description)
- Landing page URL/content
What to design
Design an end-to-end harmful content detection system that:
- Screens ads at submission time (and/or pre-serve) with low latency.
- Supports high throughput (large ad volume).
- Provides a decision: approve / reject / send to human review .
- Handles ambiguous cases, adversarial attempts, and policy changes.
- Enables continuous improvement via feedback and retraining.
Deliverables
In your design, cover:
- Functional + non-functional requirements (latency, throughput, availability)
- High-level architecture and key components
- Model approach (multi-modal), feature extraction, inference serving
- Data labeling, human review workflow, and feedback loops
- Monitoring/metrics and quality evaluation (precision/recall, false positives)
- Safety, privacy, and abuse considerations
Solution
Show