Conditional probability and Bayes' rule

In a data science onsite, conditional probability usually arrives as a product question: an alert fired, a user upgraded, a check came back positive. Underneath sits one question: given what we just saw, how likely is the thing we care about? Interviewers use it to see whether you can keep two similar-sounding probabilities apart while talking. By the end of this lesson you will be able to read a conditional probability off a table, tell independent events from mutually exclusive ones, combine cases with the law of total probability, run Bayes' rule with counts out of 10,000, and update again when a second piece of evidence arrives.

Read a conditional probability off a two-way table

P(A | B), read "the probability of A given B", is the share of the outcomes in B in which A also happens. Here are 2,000 users from a free trial, split by device and by whether they upgraded to a paid plan:

Conditioning means you keep one row or one column and divide by its total.

• P(upgraded | mobile) = 60/1,200 = 5%. Keep the mobile row.

• P(mobile | upgraded) = 60/140, about 42.9%. Keep the upgraded column.

Same cell, two denominators, and two answers that differ by a factor of almost nine. The general formula makes the same move with probabilities: P(A | B) = P(A and B)/P(B). Here that is (60/2,000)/(1,200/2,000) = 60/1,200.

"Given mobile, so I'm only looking at the 1,200 mobile users. Sixty of them upgraded, so 5%."

Saying the denominator out loud costs two seconds, and it is the quickest way to notice that you have flipped a conditional.

Multiply along a chain of events

Rearranging the formula gives the multiplication rule: P(A and B) = P(B) × P(A | B). From the table, P(mobile and upgraded) = 0.6 × 0.05 = 0.03, which is 60 of the 2,000 users.

The rule chains. A batch of 10 model training runs includes 3 that leaked test data into training. You audit 3 runs chosen at random, without replacement. The probability that the audit finds no leaky run is

7/10 × 6/9 × 5/8 = 210/720 = 7/24, about 29.2%.

Each factor is conditional on the ones before it: after one clean pick, 6 of the remaining 9 runs are clean. The counting route from counting and symmetry agrees: C(7, 3)/C(10, 3) = 35/120 = 7/24. So this audit misses every leak more than a quarter of the time.

Independent is not the same as mutually exclusive

These two terms sound alike and behave in opposite ways.

• Independent: P(A and B) = P(A) × P(B). When P(B) > 0 this is the same as P(A | B) = P(A): learning that B happened does not change the probability of A. For three or more events, every pair passing this test is not enough: mutual independence needs the product rule to hold for every group of them, all of them together included.

• Mutually exclusive: A and B cannot both happen. P(A and B) = 0.

Test the table for independence, for a user drawn at random from these 2,000. Overall, P(upgraded) = 140/2,000 = 7%, but among mobile users it is 5%, so device and upgrading are not independent. The product test agrees: P(mobile) × P(upgraded) = 0.6 × 0.07 = 0.042, while the actual share is 0.03. If the 2,000 are a sample from a larger population, a test, the subject of hypothesis tests and p-values, would compare mobile's 5% with desktop's 10% (80 of 800), not with the overall 7%, which includes the mobile users. With groups this size, a gap that wide is far too large to be chance.

Now take "mobile" and "desktop". Each user sits in exactly one row, so the two events are mutually exclusive, and that makes them strongly dependent: learning a user is on mobile drops the chance they are on desktop from 40% to 0. Two events that each have some chance of happening can never be both mutually exclusive and independent.

Here is what the interviewer sees when the terms get mixed: a candidate adds the probabilities of independent events to get "A or B", or multiplies the probabilities of mutually exclusive events to get "A and B". With two independent jobs that each fail 5% of the time, adding gives 10% for "at least one fails". The right answer is 1 - 0.95^2 = 9.75%, because adding counts the case where both fail twice. Add only when the events cannot happen together. Multiply only when they are independent, or use the conditional when they are not.

Bayes' rule with 10,000 servers

Problem. In a fleet of servers, 4% have a disk that will fail this month. A monitoring alert fires for 95% of those servers, and also for 10% of the healthy ones. An alert just fired. How likely is it that this server's disk is failing?

First, the law of total probability. To get the chance of an event, split the world into cases that don't overlap and together cover everything, then average the event's chance across the cases, weighted by how common each case is. The alert fires at a rate of

P(alert) = 0.95 × 0.04 + 0.10 × 0.96 = 0.038 + 0.096 = 0.134.

The same move recovers the table's 7%: 0.05 × 0.6 + 0.10 × 0.4 = 0.07.

Bayes' rule turns a conditional around. For a hypothesis H and evidence E,

P(H | E) = P(E | H) × P(H)/P(E),

and the law of total probability supplies the bottom: P(E) = P(E | H) × P(H) + P(E | not H) × P(not H). You can test it on the table: P(mobile | upgraded) = 0.05 × 0.6/0.07, about 42.9%, the same answer as reading the column. For the alert:

P(failing | alert) = P(alert | failing) × P(failing)/P(alert) = 0.038/0.134, about 28.4%.

The formula is easy to garble at a whiteboard. Natural frequencies, the same numbers written as counts of servers out of 10,000, are much harder to get wrong:

Of the 1,340 alerts, 380 are real: 380/1,340, about 28.4%. In the terms data science interviews use, the 95% is the alert's sensitivity (recall), the 10% is its false positive rate (1 - specificity), and the 28.4% posterior is its precision (positive predictive value).

"Out of 10,000 servers, 400 are failing and the alert catches 380 of them. Ten percent of the 9,600 healthy ones also alert, which is 960. So there are 1,340 alerts and 380 are real, about 28%."

The mistake the interviewer is waiting for is the base-rate mistake: answering 95% (the alert's hit rate) or 90% (one minus its false-alarm rate). Both answers ignore how rare failing disks are. Healthy servers outnumber failing ones 24 to 1, so a 10% false-alarm rate on the 9,600 healthy servers gives 960 false alerts, more than the 380 real ones. The table also shows the other side: with no alert, P(failing) = 20/8,660, about 0.23%. This alert is much better at clearing a server than at convicting one.

The same alert means different things in different fleets:

The alert is equally good in every row. Only the base rate changes, and it moves the answer from under 10% to over 70%. If an interviewer leaves the base rate out of a question like this, ask for it before computing anything.

For mental math, use odds, the ratio of "yes" to "no". The prior odds of a failing disk are 400 to 9,600, or 1 to 24. The likelihood ratio of the alert is how much more often it fires on failing servers than on healthy ones: 0.95/0.10 = 9.5. Multiply the odds by it: 9.5 to 24, which is a probability of 9.5/33.5, about 28.4%.

Update a second time

Alerted servers go through a slower diagnostic. It flags 80% of failing disks and 5% of healthy ones, and its mistakes are unrelated to the alert's once you know the disk's true state. The term for that is conditionally independent. The diagnostic flags this server. Now what?

Start from the 1,340 alerted servers, not from all 10,000. The posterior after the first piece of evidence is the prior for the second.

In odds, the diagnostic's likelihood ratio is 0.80/0.05 = 16, so 9.5 to 24 becomes 152 to 24, and 152/176 is about 86.4%. Each conditionally independent piece of evidence multiplies the odds by its own likelihood ratio, so the order does not matter: running the diagnostic first and the alert second ends at the same 86.4%. Had the diagnostic cleared the server instead, 380 × 0.20 = 76 failing and 960 × 0.95 = 912 healthy servers would remain, and P(failing) would fall to 76/988, about 7.7%.

The conditional independence assumption carries the whole update. If the diagnostic reads the same sensor as the alert, a healthy server that tripped the alert is more likely to trip the diagnostic too, and multiplying the two ratios overstates your confidence. State the assumption, and say what would break it.

A simulation checks both updates without any formula:

import numpy as np

rng = np.random.default_rng(1)
n = 1_000_000
failing = rng.random(n) < 0.04
alert = rng.random(n) < np.where(failing, 0.95, 0.10)
diagnostic = rng.random(n) < np.where(failing, 0.80, 0.05)
print(round(failing[alert].mean(), 3))               # close to 0.284 (exact 380/1,340)
print(round(failing[alert & diagnostic].mean(), 3))  # close to 0.864 (exact 304/352)

Practicing with the questions on this page

For each question, write down the event you are conditioning on before anything else, and name its denominator. When a question gives rates, turn them into counts out of 10,000 and fill a table. When it gives repeated evidence, track the odds and multiply. Then check direction: evidence that favors a hypothesis should raise its probability, and a rare hypothesis needs strong evidence before it becomes likely. If the base rate never appears in your working, start again from the table.

Next, expected value and linearity turns these probabilities into averages, counts and prices.