Design and decompose Trust & Safety risk metrics

You are a Data Scientist in a Trust & Safety team for a short-video platform (similar to TikTok/Reels).

The team asks: “How would you design risk metrics, and how would you decompose () them?”

Context

The platform faces multiple risk sources and controls:

• Risk types : policy-violating content (nudity/violence/hate), spam/scams, bot activity, account takeover, misinformation.
• Detection & enforcement signals : ML model predictions, proactive human review, reactive user reports, and enforcement actions (remove, downrank, age-gate, suspend).

Task

1. Propose a metric framework to measure “risk” at the platform level.

• Define primary metric(s) and diagnostic / guardrail metrics .
• Specify exact definitions (numerator/denominator), unit (per view/per user/per content), and time window (e.g., daily in UTC).

2. Explain what it means to decompose the risk metric (“”), and provide a concrete metric tree / breakdown that helps identify why risk increased or decreased.

• Show at least two different decomposition approaches (e.g., by funnel stage vs. by segment).

3. List the key data/measurement pitfalls and how you would address them.

• Examples: label delay, selection bias from user reports, changing enforcement policies, feedback loops from downranking/removal, duplicated content, and seasonality.

Your answer should be actionable for ongoing monitoring and root-cause analysis, not just high-level ideas.