Determine Posterior Probability of Bad User Prediction

Q: Determine Posterior Probability of Bad User Prediction

Meta Bayes theorem question on a bad-actor classifier, covering prevalence, sensitivity, specificity, false positives, positive predictive value, and the base-rate effect.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Q: What difficulty level is this interview question?

This is a easy difficulty Statistics & Math question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

Posterior Probability for a Bad-Actor Classifier

You are evaluating a binary classifier that flags bad actors among users.

Given:

5% of users are truly bad.
The model has 95% sensitivity: it correctly flags 95% of truly bad users.
The model has 95% specificity: it correctly labels 95% of truly good users as good.

If the model predicts that a user is bad, what is the posterior probability that the user is truly bad?

Constraints & Assumptions

Interpret "95% correct for both classes" as 95% sensitivity and 95% specificity.
Use Bayes' theorem.
Show the base-rate effect.
Report the result as a probability or percentage.

Clarifying Questions to Ask Guidance

Does 95% accuracy mean class-wise sensitivity and specificity, or only overall accuracy?
Is the 5% prevalence measured in the same population where the classifier is deployed?
Are there different costs for false positives and false negatives?

What a Strong Answer Covers Guidance

Define events for truly bad, truly good, and predicted bad.
Use P(Bad | Predicted Bad) = P(Predicted Bad | Bad)P(Bad) / P(Predicted Bad) .
Include false positives from the 95% good-user population.
Compute numerator 0.95 * 0.05 .
Compute denominator 0.95 * 0.05 + 0.05 * 0.95 .
Conclude that the posterior probability is 50%.
Explain why high class-wise accuracy can still produce only 50% precision when prevalence is low.

Follow-up Questions Guidance

What happens if prevalence falls to 1%?
How would you choose a threshold if false positives are very costly?
Which metric is this posterior probability also known as?

Determine Posterior Probability of Bad User Prediction

Quick Overview

Determine Posterior Probability of Bad User Prediction

Posterior Probability for a Bad-Actor Classifier

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer