## Context You are a Data Scientist on the integrity / harmful-content team for a large social media product. Leadership wants a way to track how bad policy-violating content is on the platform over time, and to evaluate the effect of integrity interventions (ranking changes, enforcement actions, classifier launches, human review). They have asked you to design a **"severity" metric — or metric suite —** for harmful content. You have access to: - **Impression / view logs** (who saw what, on which surface). - **Content metadata** (creator, surface, modality, language, region). - **Policy labels from human review** — accurate but only **partial coverage**. - **ML classifier scores** — full coverage but imperfect precision/recall. Interventions can change both the **amount** of violating content created and the **distribution of views** across content. ### Constraints & Assumptions - "Violating content" must be defined by you; a reasonable default is content confirmed policy-violating by human review, or scored above a high-confidence classifier threshold. - Human-review labels cover only a small, possibly non-random sample of all content; classifier coverage is complete but biased (e.g., weaker in low-resource languages). - Violation types span a wide severity range (e.g., low-level spam vs. self-harm or violent-threat content). - The metric will be reported regularly to leadership and used to gate intervention launches, so it must be both interpretable and hard to game. - The platform spans many regions, languages, and surfaces (feed, video, stories, DMs), and traffic mix shifts over time and with product launches. ### Clarifying Questions to Ask - What is the metric **for** — executive reporting, intervention evaluation, on-call alerting, or all three? (Different uses favor different stability/sensitivity tradeoffs.) - Is the goal a **single north-star number** or are we allowed a small suite (primary + diagnostics + guardrails)? - What is the **policy taxonomy and severity tiering** — do we already have agreed harm weights per violation class, or do we need to propose them? - What is the **ground-truth labeling pipeline** — sampling design, coverage per segment, and label latency? - Which **surfaces and modalities** are in scope, and is impression logging consistent across them (feed vs. DMs vs. stories vs. video watch time)? - Do we need the metric to support **causal** evaluation of interventions (A/B tests available?) or only trend monitoring? ### Part 1 — Propose metrics for harmful-content severity Propose a set of metrics to measure the severity of violating/harmful content on the platform. For each metric, define what it means, specify the **unit of analysis** (content item, user, impression/view, session, day), and state how it captures a distinct facet of "severity" (exposure, breadth, intensity, downstream harm, operational response). State clearly what counts as a "violation" in your definitions. ```hint Decompose "severity" first "Severity" is not one thing. Before listing metrics, name the distinct dimensions you want to cover — e.g. **exposure** (how many views), **breadth** (how many distinct users), **intensity** (how serious each violation is), **downstream harm** (reports/blocks/churn), and **operational response** (time-to-action). Map metrics onto these. ``` ```hint Exposure vs. supply vs. severity-weighting Distinguish *supply* metrics (what gets created: violating-post rate) from *exposure* metrics (what gets seen: view/user prevalence). Then ask how to fold in seriousness — a **severity weight** $w_i$ per violation class turns a flat ratio into $\frac{\sum_i w_i \cdot \text{views}_i}{\text{total views}}$. ``` ```hint Handle partial labels You don't know ground truth for every viewed item. Think about how a **calibrated classifier score** $p(\text{violation}\mid x)$ plus **stratified human-audit sampling** lets you *estimate* prevalence rather than only counting confirmed labels. ``` #### What This Part Should Cover - A small set of metrics spanning **multiple severity dimensions** (not five flavors of the same ratio): exposure, breadth, intensity/severity-weighting, downstream-harm proxy, operational/time-to-action. - A precise, defensible definition of "violation" and how partial labeling + classifier scores feed each metric. - Correct, explicit **units of analysis** per metric (per-view, per-user-week, per-DAU, etc.) and awareness that the unit changes what the metric answers. ### Part 2 — Pros and cons of View Prevalence The team suggests **View Prevalence (VP)** as the main KPI: $$VP = \frac{\text{views/impressions of violating content}}{\text{all views/impressions}}$$ Discuss the pros and cons of View Prevalence as a primary severity metric. ```hint Stress the denominator and what it hides A ratio moves when *either* side changes. Ask: what happens to VP when total (non-violating) views surge from a product launch, even if violating views are flat? And does a single ratio reveal whether harm is **concentrated** in a few heavy-exposure users vs. **spread** thinly across many? ``` ```hint Measurement vs. reality, and gaming VP is computed from labels you don't fully have. Consider how a **classifier precision/recall shift** can move VP with no change in actual harm, and how a target on VP can be hit by reducing total exposure or pushing content to less-instrumented surfaces (Goodhart). ``` #### What This Part Should Cover - Genuine **strengths**: direct exposure framing, normalized comparability over time, sensitivity to ranking/enforcement, easy to communicate. - Concrete **failure modes**: denominator artifacts, concentration-vs-breadth blindness, severity blindness, label/measurement bias from classifier drift, Goodhart/gaming, Simpson's paradox across segments, non-causality (seasonality/news/adversarial spikes). - A clear verdict on whether VP should be the *sole* KPI. ### Part 3 — Key tradeoffs Discuss the key tradeoffs involved in choosing and optimizing these metrics. At minimum cover: **user safety vs. engagement**, **precision vs. recall**, **reporting robustness vs. sensitivity to change**, and **fairness/coverage across regions and languages**. ```hint Make each tradeoff actionable For each tension, don't just name it — say which **proxy** you'd measure on each side and how you'd resolve it. E.g. precision↔recall: pair an overturn/appeal rate (precision proxy) with a backfill-audit or "found-via-report share" (recall proxy). ``` #### What This Part Should Cover - Each named tradeoff articulated as a real tension with a **mechanism** (not a slogan), plus the metric/proxy you'd watch on each side. - A position on how to *resolve* tensions: guardrails for safety-vs-engagement, segmented reporting + minimum-coverage for fairness, layered metrics for robustness-vs-sensitivity. - Awareness of second-order effects: supply-vs-exposure optimization, short-vs-long-term feedback loops, evasion/migration. ### Part 4 — Recommend a final metric suite If asked to recommend a final metric suite, state **which metric you'd pick as the primary KPI**, and what your **diagnostic** and **guardrail** metrics would be. Justify the choice. ```hint Primary should fix VP's worst flaws Pick a primary that keeps VP's exposure intuition but repairs its severity-blindness — a **severity-weighted** or expected-harm exposure metric. Then use diagnostics to catch what a single ratio hides (breadth, tails, segments) and guardrails to prevent gaming the KPI at the expense of users. ``` #### What This Part Should Cover - A clearly designated **primary KPI** with rationale, ideally a severity-weighted exposure metric, plus an implementation path (rule-based tiers → calibrated scoring). - A coherent **diagnostic** layer (user prevalence, per-DAU normalization, segmented breakdowns, tail/p95 exposure, time-to-action) that explains *why* the primary moved. - A **guardrail** layer covering false-positive harm (appeals/overturns), core product health, and measurement health (coverage, calibration drift, logging completeness). ### What a Strong Answer Covers Across all parts, a strong answer treats "severity" as multi-dimensional and resists collapsing it to one naive ratio without acknowledging the cost. Beyond the per-part rubrics, look for: - **Coherence across parts**: the chosen primary KPI in Part 4 is a direct, deliberate response to the VP weaknesses raised in Part 2 and the tradeoffs in Part 3. - **Measurement rigor**: the candidate separates *detection changes* from *real-world changes*, uses stratified sampling + reweighting to estimate true prevalence, and treats classifier calibration as a first-class concern. - **Operating-the-metric mindset**: how it would be reported (cadence, segmentation), how launches are gated, and how to evaluate interventions causally (prefer A/B; otherwise diff-in-diff / synthetic control with named confounders). - **Product/safety judgment**: never optimizes a single number into a hollow "win"; explicitly protects against Goodhart and fairness blind spots. ### Follow-up Questions - A new classifier increases recall by 15% in Spanish and Portuguese. Reported severity-weighted VP **rises**. How do you tell leadership whether the platform got worse or your measurement got better — and what do you report? - You must alert on-call when harm spikes. Would you alert on the weekly primary KPI, or design a separate signal? What would the alerting metric and threshold logic look like? - How would you set the **severity weights** $w_i$ across policy classes in a defensible, non-arbitrary way, and how would you detect when they're driving misleading movements in the KPI? - An intervention reduces VP today but you suspect creators are migrating to a less-instrumented surface (e.g. DMs). How would you detect and account for this displacement?