Measuring User Exposure to Violating Content on a UGC Platform

Context

You work on a large-scale user-generated content (UGC) platform that uses automated and human moderation. You need a robust metric framework to quantify and monitor how much violating content users are exposed to, and to evaluate interventions that reduce this exposure.

Assumptions (define or adapt as needed):

• A "view" is a content render that meets viewability thresholds (e.g., visible ≥1s, ≥50% viewport).
• A "session" is a sequence of user activity separated by ≥30 minutes of inactivity.
• DAU is the count of distinct real human users with ≥1 session in the day.
• A "violating item" is a post/video/comment that violates policy after review (human or high-confidence auto) subject to appeals.

Tasks

1. Define precise formulas for at least three daily metrics and their 7-day rolling counterparts:
   • view_prevalence = violating_views / total_views
   • violating_session_rate = sessions_with_≥1_violating_view / total_sessions
   • violations_per_active_user = violating_views / DAU
2. Specify exact inclusion/exclusion rules for what counts as a violating view under two timing regimes:
   • Ex-ante: only violations known at the time of view
   • Ex-post: final decision after reviews Include how to treat late-arriving labels, appeals, deleted content, repeat views, and bot traffic.
3. Is view_prevalence a good north-star metric? Compare it with:
   • incident_rate = violating_items / items_created
   • user_prevalence = users_exposed / DAU Discuss tradeoffs: detection lag, denominator gaming, precision/recall shifts, Simpson’s paradox across countries/surfaces, and Goodhart’s law.
4. Propose a weekly alerting method with thresholds using uncertainty estimates (e.g., Wilson or Bayesian beta–binomial intervals). Describe guardrail metrics (e.g., false positive exposure, creator churn, review queue SLA).
5. Sketch an A/B test to reduce view_prevalence: state the primary metric, key segments (country, surface, creator cohort), power assumptions, and how you’ll correct for label latency and selection bias when violations are discovered after exposure.