Posterior probability given model accuracy

Q: Posterior probability given model accuracy

Evaluates Bayesian posterior probability for a flagged bad-user classifier with low base rate and symmetric 95% sensitivity and specificity. Strong answers compute precision with Bayes' theorem and explain the base-rate effect.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a easy difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

Security Classification: Posterior Probability When Flagged

You are evaluating a binary classifier that flags potentially bad users.

Assume:

Base rate of bad users: P(Bad) = 5%
Sensitivity, or true positive rate: P(Flagged | Bad) = 95%
Specificity, or true negative rate: P(Not Flagged | Good) = 95%

Using Bayes' theorem, compute P(Bad | Flagged).

Constraints & Assumptions

Treat accuracy as symmetric sensitivity and specificity as stated.
Show the false positive rate.
Use Bayes' theorem and explain the base-rate effect.
Report the final probability clearly.

Clarifying Questions to Ask Guidance

Does "95% accuracy" mean both sensitivity and specificity are 95%?
What is the base rate in the scored population?
Are flagged users all reviewed, blocked, or just marked for investigation?
Is the goal precision, recall, or cost-sensitive decisioning?

What a Strong Answer Covers Guidance

Defines prior P(Bad) and P(Good).
Uses TPR = 0.95 and FPR = 0.05.
Computes numerator as 0.95 times 0.05.
Computes denominator as true positives plus false positives.
Gives P(Bad | Flagged) = 0.5 under the stated assumptions.
Explains why high sensitivity and specificity can still produce moderate precision when base rate is low.

Follow-up Questions Guidance

How would the posterior change if the base rate were 1%?
What if specificity improves to 99%?
Which metric matters more if false positives are very costly?

Posterior probability given model accuracy

Quick Overview

Posterior probability given model accuracy

Security Classification: Posterior Probability When Flagged

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer