How would you measure misinformation impact and recommendation bias?

You are interviewing for an experienced Data Scientist role on a short-form video platform (e.g., TikTok). Product sense / case questions come up frequently. ## Case A — Estimate “bad AIGC / fake news” impact in 1 day The platform has limited content reviewers. Leadership asks you to use **one day** to measure the **impact of fake news / low-quality AIGC (“bad AIGC”)** on the platform. **Task:** Propose an end-to-end plan to estimate (1) how much “bad AIGC/fake news” exists and (2) its user impact, under tight time and labeling constraints. Include: - A working definition of “bad AIGC/fake news” and how you operationalize it. - What metrics you would report (primary + diagnostics + guardrails). - How you would sample content and users to get an estimate quickly. - How you would quantify uncertainty (e.g., confidence intervals) and known limitations. - How you would avoid common biases (selection, survivorship, reviewer inconsistency). ## Case B — Validate claim of confirmation bias for minor users Mainstream media claims there is **confirmation bias** in the recommendation system **for minor (underage) users**. **Task:** Design an analysis/experiment to validate or refute the claim. Include: - How you define “confirmation bias” in a recommender context (measurable). - What comparison baselines/counterfactuals you would use. - How you would handle confounding (e.g., user preference vs algorithm effect), selection bias, and Simpson’s paradox. - What slices you would analyze (age bands, geography, onboarding stage, etc.). - What actions you would recommend depending on outcomes (mitigations, monitoring).