End-to-End ML System Design: Flag Illegal YouTube Videos

You are tasked with designing a production ML system to detect and triage potentially illegal YouTube videos at scale. The system must work across modalities (vision, audio, text), handle sparse/noisy labels, strong class imbalance, evolving policies, and integrate with human review.

Assumptions (make minimal, explicit):

• "Illegal" follows platform policy (e.g., child safety, terror content, incitement to violence), with versioned policy definitions that evolve over time.
• Actions include: automatic block, downrank/age-restrict, route to human review, or allow.
• The system must support multilingual/global content and near-real-time decisions.

Design the system across the following areas:

1) Data

• Inputs: video frames/thumbnails, audio tracks, ASR captions/transcripts, titles/descriptions/tags, uploader/channel metadata, user flags, policy takedown logs.
• Constraints: sparse and noisy labels, severe class imbalance, evolving policies.
• Describe data ingestion, feature storage, deduplication/near-duplicate handling, label pipelines (including policy-version tracking), and privacy/retention considerations.

2) Modeling

• Choose architectures per modality (vision, audio, text) and a multimodal fusion approach.
• Pretraining/embeddings strategy (self-supervised/foundation models; multilingual coverage).
• Strategy for weak supervision (heuristics, user flags, external lists) and active learning to acquire high-value labels.
• Handling class imbalance, noisy labels, and continual learning under policy drift.

3) Evaluation

• Offline metrics: AUROC, PR-AUC (class imbalance), calibration (ECE/Brier), and cost-weighted utility.
• Thresholding for triage tiers (auto-block, send-to-review, allow), grounded in expected utility and reviewer capacity.
• Build a reliable test set that resists leakage, near-duplicates, and distribution shift; include slice-based evaluation (language, region, topic, channel age).

4) Safety and Abuse Resistance

• Anticipate adversarial evasion and propose robustification and monitoring (without revealing evasion recipes).
• Fairness and false-positive harm mitigation; transparent appeals workflow; reversibility of actions.
• Human-in-the-loop design: reviewer tooling, quality control, throughput/SLA constraints, and prioritization.

5) Online Rollout and Measurement

• Rollout plan: shadow mode, canary, progressive ramp, and interleaving with existing human/rule systems; kill switches.
• Counterfactual risk estimation using IPS/DR to estimate violation risks and action costs offline.
• Experiment design to measure reduction in policy violations without selection bias; randomized auditing to estimate true prevalence.