Design a Harmful-Content Identification System
Company: Snapchat
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
Design a system that identifies harmful content. Define the modeling problem, labels, evaluation, and deployment workflow before choosing a model.
### Constraints and Clarifying Questions
The report does not specify content modality, harm categories, action policy, or scale. For a concrete baseline, choose uploaded videos with text metadata and a reviewed set of policy categories. These are practice assumptions. Clarify whether the system flags content for review, restricts distribution, or takes another action; detection scores alone do not define policy.
### What a Strong Answer Covers
A concrete labeling process and model, severity-aware evaluation, uncertainty and human review, multimodal serving, adversarial or distribution changes, and an appeal or correction feedback path.
### Follow-up Questions
- Why can aggregate accuracy hide failures on rare severe categories?
- How would you evaluate a threshold change without confusing detection quality with enforcement policy?
Overview: Design a concrete multimodal harmful-content classifier and review workflow. Define labels, masked losses, severity-aware thresholds, independent evaluation, and feedback from corrections and appeals.
Read the full Snapchat Machine Learning Engineer interview experience this question came from