Measuring and mitigating fake news on Facebook

Q: Measuring and mitigating fake news on Facebook

Evaluates fake-news prevalence measurement under limited reviewer capacity and model-assisted sampling. Strong answers use model-assisted sampling, content and exposure prevalence, confidence intervals, ongoing measurement, model improvement, and mitigation.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

Measuring and Mitigating Fake News Under Reviewer Constraints

Policy teams need an overnight view of fake-news prevalence on the platform, but only a small number of human reviewers are available. Leadership also wants a long-term statistically sound measurement program and a plan to improve detection models.

Report both content prevalence, the percentage of posts that are fake, and exposure prevalence, the percentage of user impressions on fake content.

Constraints & Assumptions

Human labels are limited and costly.
An existing ML model can pre-score content as likely fake or not.
Impression counts are available to estimate exposure.
Include uncertainty bounds and sampling bias corrections.

Clarifying Questions to Ask Guidance

What policy definition determines "fake news"?
What content universe and time window are in scope?
Are impressions, reshares, and cross-posted content deduped?
How reliable are existing model scores and reviewer labels?

Part 1 - Rapid Overnight Measurement

With limited reviewers, how would you estimate fake-news impact within a single day?

What This Part Should Cover Guidance

Use stratified or model-assisted sampling based on ML risk scores.
Allocate reviews across high-, medium-, and low-risk strata.
Weight labels back to the full content and impression universe.
Report content prevalence, exposure prevalence, and confidence intervals.

Part 2 - Long-term Measurement Program

How would you build a statistically sound ongoing measurement system?

What This Part Should Cover Guidance

Use recurring random and stratified samples, reviewer quality control, adjudication, and calibration.
Track prevalence by content type, geography, language, source, and distribution channel.
Monitor trend breaks and uncertainty over time.
Separate measurement labels from model-training labels where needed.

Part 3 - Model Improvement and Mitigation

How would you improve detection models and reduce exposure?

What This Part Should Cover Guidance

Use labeled data, active learning, threshold tuning, model calibration, and human review queues.
Optimize for exposure reduction, precision, recall, appeal quality, and policy costs.
Test ranking demotion, warning labels, friction, removal, or sharing limits.
Monitor false positives, fairness, and adversarial adaptation.

Follow-up Questions Guidance

What if the model misses a low-score cluster of fake news?
How would you choose between content prevalence and exposure prevalence for executives?
How would you prevent reviewer bottlenecks from biasing measurement?

Measuring and mitigating fake news on Facebook

Quick Overview

Measuring and mitigating fake news on Facebook

Measuring and Mitigating Fake News Under Reviewer Constraints

Constraints & Assumptions

Clarifying Questions to Ask Guidance

Part 1 - Rapid Overnight Measurement

What This Part Should Cover Guidance

Part 2 - Long-term Measurement Program

What This Part Should Cover Guidance

Part 3 - Model Improvement and Mitigation

What This Part Should Cover Guidance

Follow-up Questions Guidance

Write your answer