PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Design metrics for violating content exposure

Last updated: Mar 29, 2026

Quick Overview

This question evaluates competency in metric design and measurement for content safety, including defining exposure metrics, inclusion/exclusion rules, handling label latency and appeals, uncertainty-aware alerting, and A/B test design for a Data Scientist role in the Analytics & Experimentation domain.

  • hard
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Design metrics for violating content exposure

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

You’re working on a UGC platform with automated and human moderation. Design a metric framework to measure user exposure to violating content. 1) Define precise formulas for at least three daily metrics and 7-day rolling counterparts: view_prevalence = violating_views / total_views, violating_session_rate = sessions_with_≥1_violating_view / total_sessions, and violations_per_active_user = violating_views / DAU. 2) Specify exact inclusion/exclusion rules for what counts as a violating view under two timing regimes: ex-ante (only violations known at view time) vs. ex-post (final decision after reviews), and how to treat late-arriving labels, appeals, deleted content, repeat views, and bot traffic. 3) Is view_prevalence a good north-star? Compare it vs. incident_rate (violating_items / items_created) and user_prevalence (users_exposed / DAU); discuss tradeoffs like detection lag, denominator gaming, precision/recall shifts, Simpson’s paradox across countries/surfaces, and Goodhart’s law. 4) Propose a weekly alerting method with thresholds using uncertainty estimates (e.g., Wilson or Bayesian beta-binomial intervals) and describe guardrail metrics (false positive exposure, creator churn, review queue SLA). 5) Sketch an A/B test to reduce view_prevalence: state the primary metric, key segments (country, surface, creator cohort), power assumptions, and how you’ll correct for label latency and selection bias when violations are discovered after exposure.

Quick Answer: This question evaluates competency in metric design and measurement for content safety, including defining exposure metrics, inclusion/exclusion rules, handling label latency and appeals, uncertainty-aware alerting, and A/B test design for a Data Scientist role in the Analytics & Experimentation domain.

Related Interview Questions

  • Measure scheduled posts feature success - Meta (medium)
  • Estimate ads ranking revenue impact - Meta (medium)
  • How should you evaluate unconnected content? - Meta (medium)
  • Should WhatsApp launch group calls? - Meta (medium)
  • How would you grow Meta products? - Meta (medium)
Meta logo
Meta
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
1
0

Measuring User Exposure to Violating Content on a UGC Platform

Context

You work on a large-scale user-generated content (UGC) platform that uses automated and human moderation. You need a robust metric framework to quantify and monitor how much violating content users are exposed to, and to evaluate interventions that reduce this exposure.

Assumptions (define or adapt as needed):

  • A "view" is a content render that meets viewability thresholds (e.g., visible ≥1s, ≥50% viewport).
  • A "session" is a sequence of user activity separated by ≥30 minutes of inactivity.
  • DAU is the count of distinct real human users with ≥1 session in the day.
  • A "violating item" is a post/video/comment that violates policy after review (human or high-confidence auto) subject to appeals.

Tasks

  1. Define precise formulas for at least three daily metrics and their 7-day rolling counterparts:
    • view_prevalence = violating_views / total_views
    • violating_session_rate = sessions_with_≥1_violating_view / total_sessions
    • violations_per_active_user = violating_views / DAU
  2. Specify exact inclusion/exclusion rules for what counts as a violating view under two timing regimes:
    • Ex-ante: only violations known at the time of view
    • Ex-post: final decision after reviews Include how to treat late-arriving labels, appeals, deleted content, repeat views, and bot traffic.
  3. Is view_prevalence a good north-star metric? Compare it with:
    • incident_rate = violating_items / items_created
    • user_prevalence = users_exposed / DAU Discuss tradeoffs: detection lag, denominator gaming, precision/recall shifts, Simpson’s paradox across countries/surfaces, and Goodhart’s law.
  4. Propose a weekly alerting method with thresholds using uncertainty estimates (e.g., Wilson or Bayesian beta–binomial intervals). Describe guardrail metrics (e.g., false positive exposure, creator churn, review queue SLA).
  5. Sketch an A/B test to reduce view_prevalence: state the primary metric, key segments (country, surface, creator cohort), power assumptions, and how you’ll correct for label latency and selection bias when violations are discovered after exposure.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.