Design a harmful video content moderation system
Company: OpenAI
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: hard
Interview Round: Onsite
## Question
Design an end-to-end system to detect and moderate harmful videos on a large platform.
## Requirements
- Detect multiple policy categories (violence, self-harm, hate, sexual content, etc.).
- Operate at upload time and for already-published content.
- Combine automated decisions with human review.
- Minimize both false negatives (missed harmful content) and false positives (wrongful removals).
- Provide appeals and auditability.
## Deliverables
Architecture, model strategy (multi-modal), labeling/review workflow, metrics, and how you handle adversarial evasion.
Quick Answer: This question evaluates a candidate's competency in designing scalable, multi-modal machine learning systems for content moderation, covering architecture, model strategy, labeling and human-in-the-loop workflows, metrics, auditability, and defenses against adversarial evasion.