How to measure harmful-content severity and run experiments

## Context You are a Data Scientist on a social media platform working on **harmful content** (e.g., hate/harassment, self-harm, violence, sexual exploitation, misinformation). A team proposes a new intervention (e.g., ranking demotion, removal policy change, or a new ML classifier + enforcement workflow). ## Task 1. **Define “severity of harmful content.”** - Propose a severity framework that supports both measurement and decision-making. - Explain tradeoffs (e.g., interpretability vs. sensitivity, policy alignment, subjectivity, multilingual considerations). 2. **Design metrics (primary/diagnostic/guardrails).** - Provide at least: - One **primary** metric capturing harmful content impact. - Several **diagnostic** metrics that help explain movement. - Several **guardrail** metrics to detect unintended harm. - Specify exact definitions (numerators/denominators) and any weighting (e.g., by exposure). 3. **Design an experiment to evaluate the intervention.** - Choose an appropriate **randomization unit** (viewer/user, author, content item, session, community/cluster, geo, etc.) and justify it. - Discuss common pitfalls: interference/spillover, network effects, contamination, novelty effects, delayed outcomes, and measurement error. - Explain how you would analyze results (e.g., intent-to-treat vs. per-protocol, variance reduction, segmentation, multiple testing). 4. **Pros/cons and edge cases.** - Highlight failure modes: label noise, policy changes during the test, adversarial behavior, reporting bias, and differences across regions/languages. ### Assumptions - You can log impressions/views, engagement, reports, enforcement actions, and model outputs. - “Harmful content” is determined by a combination of policy rules, human review, and ML signals (imperfect).