Compute fraud probabilities with Bayes and Binomial
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: medium
Interview Round: Onsite
An online platform classifies accounts as fake or authentic. Prior: 3% of accounts are fake.
Signals: For each account in the last week there are n = 5 independent sessions. In each session, a “suspicious action” occurs with probability p_F = 0.5 if the account is fake and p_A = 0.05 if authentic. The rule flags an account if it has at least k suspicious sessions.
(a) For k = 2, compute TPR = P(flag | fake) and FPR = P(flag | authentic) using the Binomial distribution. Show formulas and numeric values.
(b) Using Bayes’ Theorem, compute PPV = P(fake | flag) and NPV = P(authentic | not flagged) for k = 2.
(c) Now a manual review is applied only to flagged accounts. The reviewer independently has sensitivity 0.90 and specificity 0.98. An account is actioned only if both the rule flags it and the reviewer says “fake.” Compute the new overall TPR and FPR, and the revised PPV.
(d) For a population of 1,000,000 accounts, compute expected counts of true positives, false positives, true negatives, and false negatives under the process in (c).
(e) For k ∈ {1,2,3,4,5}, which k maximizes F1 score on the prior above without the manual review step? Outline the computation and provide the numeric choice. Discuss how the optimal k would change if the base fake rate rose to 10%.
(f) Identify which errors in (a)–(e) correspond to Type I vs. Type II errors in this context.
Overview: This question evaluates a candidate's understanding of probabilistic modeling and statistical decision-making, focusing on the Binomial distribution for session-level events and Bayes' theorem for posterior probabilities in a fraud-detection setting.
Read the full Meta Data Scientist interview experience this question came from
Community answers
Answer by SS
You have accounts on a platform. Each account is either:
Fake (probability of suspicious action per session = 0.50)
Authentic (probability of suspicious action per session = 0.05)
Each account had 5 sessions last week. You watch how many of those 5 sessions had a suspicious action.
Your detector flags an account if at least 2 out of 5 sessions are suspicious (k = 2).
The question is: how good is this rule at catching fakes while leaving authentic accounts alone?
What is TPR and FPR in plain English?
TPR (True Positive Rate) = if an account is genuinely fake, what's the chance your rule catches it? You want this HIGH.
FPR (False Positive Rate) = if an account is genuinely authentic, what's the chance your rule wrongly flags it? You want this LOW.
Why Binomial?
Each session is independent, and in each session the suspicious action either happens or doesn't. That's exactly the Binomial setup — like flipping a coin 5 times and counting heads.
The Binomial formula for getting exactly k suspicious sessions out of n = 5:
P(X = k) = C(n, k) × p^k × (1−p)^(n−k)
Where C(n,k) is "n choose k" — the number of ways to pick k sessions out of n.
"At least 2" means: P(X ≥ 2) = 1 − P(X = 0) − P(X = 1)
It's easier to subtract the "fewer than 2" cases from 1 than to add up all the "2 or more" cases.
Computing TPR — for a FAKE account (p = 0.50)
First find P(X = 0) and P(X = 1):
P(X = 0) = C(5,0) × 0.5⁰ × 0.5⁵ = 1 × 1 × 0.03125 = 0.03125
P(X = 1) = C(5,1) × 0.5¹ × 0.5⁴ = 5 × 0.5 × 0.0625 = 0.1562
Answer by SS
Let's solve this step by step, building on the numbers from part (a).
What we already know
Prior fake rate: P(fake) = 0.03 (3%)
So P(authentic) = 0.97 (97%)
TPR = P(flag | fake) = 0.8125
FPR = P(flag | authentic) = 0.0226
Imagine 10,000 accounts to make it concrete
This is the easiest way to see what's happening before doing any formula.
Fake accounts = 3% × 10,000 = 300
Authentic accounts = 97% × 10,000 = 9,700
Now apply the detector:
Fake accounts flagged (true positives) = 81.25% × 300 = 243.75 ≈ 244
Fake accounts missed (false negatives) = 300 − 244 = 56
Authentic accounts flagged (false positives) = 2.26% × 9,700 = 219.22 ≈ 219
Authentic accounts not flagged (true negatives) = 9,700 − 219 = 9,481
So out of 10,000 accounts:
Total flagged = 244 + 219 = 463
Total not flagged = 56 + 9,481 = 9,537
PPV — Positive Predictive Value
PPV = P(fake | flag) = "of all flagged accounts, how many are actually fake?"
Using Bayes:
P(fake | flag) = P(flag | fake) × P(fake) / P(flag)
First find P(flag) — the total probability of being flagged:
P(flag) = P(flag | fake) × P(fake) + P(flag | authentic) × P(authentic)
= 0.8125 × 0.03 + 0.0226 × 0.97
= 0.024375 + 0.021922
= 0.046297
Now plug in:
PPV = 0.8125 × 0.03 / 0.046297
= 0.024375 / 0.046297
= 0.5265 (52.65%)
From the concrete numbers: 244 / 463 = 52.7% ✓
In plain English: just over half of flagged accounts are actually fake. Nearly half are innocent accounts wrongly flagged.
NPV — Negative Predictive Value
NPV = P(authentic | not flagged)