Design a harmful video content moderation system

Quick Overview

This question evaluates a candidate's competency in designing scalable, multi-modal machine learning systems for content moderation, covering architecture, model strategy, labeling and human-in-the-loop workflows, metrics, auditability, and defenses against adversarial evasion.

Design a harmful video content moderation system

Company: OpenAI

Role: Machine Learning Engineer

Category: ML System Design

Difficulty: hard

Interview Round: Onsite

## Question Design an end-to-end system to detect and moderate harmful videos on a large platform. ## Requirements - Detect multiple policy categories (violence, self-harm, hate, sexual content, etc.). - Operate at upload time and for already-published content. - Combine automated decisions with human review. - Minimize both false negatives (missed harmful content) and false positives (wrongful removals). - Provide appeals and auditability. ## Deliverables Architecture, model strategy (multi-modal), labeling/review workflow, metrics, and how you handle adversarial evasion.

Quick Answer: This question evaluates a candidate's competency in designing scalable, multi-modal machine learning systems for content moderation, covering architecture, model strategy, labeling and human-in-the-loop workflows, metrics, auditability, and defenses against adversarial evasion.

|Home/ML System Design/OpenAI
OpenAI logo
OpenAI
Dec 15, 2025, 12:00 AM
hardMachine Learning EngineerOnsiteML System Design
8
0

Question

Design an end-to-end system to detect and moderate harmful videos on a large platform.

Requirements

  • Detect multiple policy categories (violence, self-harm, hate, sexual content, etc.).
  • Operate at upload time and for already-published content.
  • Combine automated decisions with human review.
  • Minimize both false negatives (missed harmful content) and false positives (wrongful removals).
  • Provide appeals and auditability.

Deliverables

Architecture, model strategy (multi-modal), labeling/review workflow, metrics, and how you handle adversarial evasion.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...