How would you measure and experiment on harmful content?

## Context You are a Data Scientist working on integrity/safety for a large user-generated content platform (e.g., social feed, short video, comments). The team is launching an intervention to reduce **harmful content** exposure (could be a new detection model, ranking demotion, warning interstitial, or stricter enforcement policy). “Harmful content” is broad and can vary in **severity** (e.g., mild harassment vs. credible threats). Labels may come from human review, user reports, and/or automated classifiers. ## Task 1. **Define “severity of harmful content.”** - Propose a practical way to quantify severity at the *content level* and then aggregate to *user*, *session*, or *platform* levels. - State assumptions about labeling sources and uncertainty. 2. **Choose metrics (with tradeoffs).** - Propose a set of **primary**, **diagnostic**, and **guardrail** metrics. - Cover both safety outcomes (harm reduction) and product outcomes (engagement/creator impact), and discuss pros/cons. 3. **Design an experiment to evaluate the intervention.** - Specify the **randomization unit** (e.g., viewer-user, creator, content item, session, geo) and justify. - Describe how you would handle: interference/spillovers, repeat exposures, cold-start/new content, delayed labels, and policy enforcement workflows. - Explain how you would estimate impact and interpret results (including failure modes and what you’d do if metrics conflict). ## Deliverables - A written plan describing severity definition, metric suite, and experiment design choices. - Include any key equations or aggregation logic needed to make the proposal implementable.