Calculate Posterior Probability of Flagged User Being Bad Actor

Quick Overview

Evaluates Bayesian inference and confusion-matrix reasoning for abuse detection. Strong answers derive posterior precision from prevalence, TPR, and FPR, define Type I and Type II errors, and compute true positives and false positives for 10 million users.

Calculate Posterior Probability of Flagged User Being Bad Actor

Company: Meta

Role: Data Scientist

Category: Statistics & Math

Difficulty: easy

Interview Round: Onsite

##### Scenario Platform must distinguish good users from potential bad actors using Bayesian inference while controlling error rates. ##### Question Given a prior probability p of a user being a bad actor and classifier outputs with known true-positive and false-positive rates, derive the posterior probability that a flagged user is truly bad. Define Type I and Type II errors in this context and explain their business impact. If 1 % of 10 million users are truly bad and the classifier has 95 % recall and 2 % false-positive rate, calculate the expected number of bad actors caught and the expected number of good users incorrectly flagged. ##### Hints Apply Bayes’ theorem, build a 2×2 confusion matrix, compute expected counts from prevalence and error rates.

Quick Answer: Evaluates Bayesian inference and confusion-matrix reasoning for abuse detection. Strong answers derive posterior precision from prevalence, TPR, and FPR, define Type I and Type II errors, and compute true positives and false positives for 10 million users.

|Home/Statistics & Math/Meta
Meta logo
Meta
Jul 12, 2025, 6:59 PM
easyData ScientistOnsiteStatistics & Math
107
0

Calculate Posterior Probability of a Flagged User Being a Bad Actor

A platform runs a binary classifier that flags users who might be bad actors. You know the base rate of bad actors and the classifier's true positive and false positive rates.

Constraints & Assumptions

  • Use Bayes' theorem and state the base-rate effect.
  • Let p be the prior probability a random user is a bad actor.
  • Let TPR = P(flag | bad) and FPR = P(flag | good) .
  • For the numeric example, use 10,000,000 users, 1% bad actors, 95% recall, and 2% false-positive rate.

Clarifying Questions to Ask Guidance

  • What action follows a flag: review, warning, rate limit, or account removal?
  • Are TPR and FPR measured on a representative sample?
  • Is the base rate stable across user segments?

Part 1 - Posterior Probability

Derive the posterior probability that a flagged user is truly a bad actor, P(bad | flag), in terms of p, TPR, and FPR.

What This Part Should Cover Guidance

  • Bayes' theorem.
  • Denominator including true positives and false positives.
  • Interpretation as precision or positive predictive value.

Part 2 - Error Types

Define Type I and Type II errors in this context and explain their business impact.

What This Part Should Cover Guidance

  • Type I as falsely flagging a good user.
  • Type II as failing to flag a bad actor.
  • Impact on user trust, appeals, fraud or abuse, support load, and platform safety.

Part 3 - Expected Counts

If 1% of 10,000,000 users are truly bad, the classifier has 95% recall and 2% false-positive rate, compute the expected number of bad actors caught and good users incorrectly flagged.

What This Part Should Cover Guidance

  • Bad actors caught: 95,000 .
  • Good users incorrectly flagged: 198,000 .
  • Optional posterior precision from the counts.

What a Strong Answer Covers Guidance

A strong answer applies Bayes' theorem correctly, explains the base-rate effect, builds the confusion matrix, and ties error rates to product and enforcement decisions.

Follow-up Questions Guidance

  • What is the posterior precision in the numeric example?
  • How would the posterior change if bad-actor prevalence doubled?
  • What threshold would you require for automatic enforcement?
Loading comments...