Calculate Posterior Probability of Flagged User Being Bad Actor
Quick Overview
Evaluates Bayesian inference and confusion-matrix reasoning for abuse detection. Strong answers derive posterior precision from prevalence, TPR, and FPR, define Type I and Type II errors, and compute true positives and false positives for 10 million users.
Calculate Posterior Probability of Flagged User Being Bad Actor
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: easy
Interview Round: Onsite
##### Scenario
Platform must distinguish good users from potential bad actors using Bayesian inference while controlling error rates.
##### Question
Given a prior probability p of a user being a bad actor and classifier outputs with known true-positive and false-positive rates, derive the posterior probability that a flagged user is truly bad. Define Type I and Type II errors in this context and explain their business impact. If 1 % of 10 million users are truly bad and the classifier has 95 % recall and 2 % false-positive rate, calculate the expected number of bad actors caught and the expected number of good users incorrectly flagged.
##### Hints
Apply Bayes’ theorem, build a 2×2 confusion matrix, compute expected counts from prevalence and error rates.
Quick Answer: Evaluates Bayesian inference and confusion-matrix reasoning for abuse detection. Strong answers derive posterior precision from prevalence, TPR, and FPR, define Type I and Type II errors, and compute true positives and false positives for 10 million users.
Calculate Posterior Probability of Flagged User Being Bad Actor
Meta
Jul 12, 2025, 6:59 PM
easyData ScientistOnsiteStatistics & Math
107
0
Calculate Posterior Probability of a Flagged User Being a Bad Actor
A platform runs a binary classifier that flags users who might be bad actors. You know the base rate of bad actors and the classifier's true positive and false positive rates.
Constraints & Assumptions
Use Bayes' theorem and state the base-rate effect.
Let
p
be the prior probability a random user is a bad actor.
Let
TPR = P(flag | bad)
and
FPR = P(flag | good)
.
For the numeric example, use 10,000,000 users, 1% bad actors, 95% recall, and 2% false-positive rate.
Clarifying Questions to Ask Guidance
What action follows a flag: review, warning, rate limit, or account removal?
Are TPR and FPR measured on a representative sample?
Is the base rate stable across user segments?
Part 1 - Posterior Probability
Derive the posterior probability that a flagged user is truly a bad actor, P(bad | flag), in terms of p, TPR, and FPR.
What This Part Should Cover Guidance
Bayes' theorem.
Denominator including true positives and false positives.
Interpretation as precision or positive predictive value.
Part 2 - Error Types
Define Type I and Type II errors in this context and explain their business impact.
What This Part Should Cover Guidance
Type I as falsely flagging a good user.
Type II as failing to flag a bad actor.
Impact on user trust, appeals, fraud or abuse, support load, and platform safety.
Part 3 - Expected Counts
If 1% of 10,000,000 users are truly bad, the classifier has 95% recall and 2% false-positive rate, compute the expected number of bad actors caught and good users incorrectly flagged.
What This Part Should Cover Guidance
Bad actors caught:
95,000
.
Good users incorrectly flagged:
198,000
.
Optional posterior precision from the counts.
What a Strong Answer Covers Guidance
A strong answer applies Bayes' theorem correctly, explains the base-rate effect, builds the confusion matrix, and ties error rates to product and enforcement decisions.
Follow-up Questions Guidance
What is the posterior precision in the numeric example?
How would the posterior change if bad-actor prevalence doubled?
What threshold would you require for automatic enforcement?