Calculate Posterior Probability of Flagged User Being Bad Actor
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: easy
Interview Round: Onsite
##### Scenario
Platform must distinguish good users from potential bad actors using Bayesian inference while controlling error rates.
##### Question
Given a prior probability p of a user being a bad actor and classifier outputs with known true-positive and false-positive rates, derive the posterior probability that a flagged user is truly bad. Define Type I and Type II errors in this context and explain their business impact. If 1 % of 10 million users are truly bad and the classifier has 95 % recall and 2 % false-positive rate, calculate the expected number of bad actors caught and the expected number of good users incorrectly flagged.
##### Hints
Apply Bayes’ theorem, build a 2×2 confusion matrix, compute expected counts from prevalence and error rates.
Overview: Evaluates Bayesian inference and confusion-matrix reasoning for abuse detection. Strong answers derive posterior precision from prevalence, TPR, and FPR, define Type I and Type II errors, and compute true positives and false positives for 10 million users.
Community answers
Answer by SS
Binary Classifier Analysis: Detecting Bad Actors
Given:
p = prior probability a random user is a bad actor (prevalence)
TPR (recall) = P(flag | bad)
FPR = P(flag | good)
1. Posterior Probability of True Bad Actor
P(bad | flag) = (TPR p) / (TPR p + FPR * (1 - p))
2. Type I and Type II Errors
Type I Error (False Positive): Flagging a good user as bad.
Business impact: legitimate users may get restricted or blocked, leading to user dissatisfaction or churn.
Type II Error (False Negative): Failing to flag a bad actor.
Business impact: bad actors continue harmful activity, causing fraud, spam, or loss of trust.
3. Compute Expected Counts
Total users = 10,000,000
True bad users = 1% → 100,000
Recall = 95% → TPR = 0.95
FPR = 2% → 0.02
True Positives (TP) = TPR #bad users = 0.95 100,000 = 95,000
False Positives (FP) = FPR #good users = 0.02 9,900,000 = 198,000
Posterior probability:
P(bad | flag) = TP / (TP + FP) = 95,000 / (95,000 + 198,000) ≈ 0.324 (32.4%)
Summary:
Only about 32% of flagged users are truly bad due to low prevalence.
False positives dominate because good users are far more common than bad users.
Trade-off: catching more bad actors vs. minimizing inconvenience to good users.