Bayesian Reasoning and Base Rates — Tech Interview Concept

What it is Bayesian reasoning updates a belief after seeing evidence: posterior ∝ prior (the base rate) × likelihood (how consistent the evidence is with each hypothesis). Base rates are the background prevalences you must factor in, e.g., disease prevalence or fraud frequency, before interpreting a test or model score.
Why interviewers ask about it Data Scientists routinely interpret A/B tests, classification scores, and alerting systems where rare events dominate (spam, fraud, safety violations). Teams at companies like Meta, Netflix, and Uber expect you to reason about priors, uncertainty, and decision thresholds so product calls are calibrated, not just “statistically significant.”
Core ideas to know

Posterior ∝ likelihood × prior; base rates are the priors. High accuracy can still mislead when prevalence is tiny.
Use natural frequencies (“1 in 1000” style) to compute posteriors reliably and avoid base‑rate neglect.
Positive predictive value depends on both sensitivity/specificity and prevalence; report it alongside recall/precision.
In A/B tests, ask “P(B is better by ≥ X)” and show credible intervals, not just p-values.
Check robustness to priors; perform sensitivity analyses and justify any informative prior.
For rare classes, set thresholds via expected loss/cost and business priors, not default 0.5 cutoffs.
Hierarchical Bayes borrows strength across segments, stabilizing metrics for low-traffic cohorts.

A common pitfall Candidates treat an impressive model metric or a “significant” experiment as decisive without considering how rare the underlying event or true lift is. With low base rates, even tiny false-positive rates can swamp true positives, leading to overreacting to noise (e.g., fraud bans, safety takedowns, or premature experiment stops). Another frequent miss is ignoring prior evidence (historical tests, seasonality) when choosing priors, producing unstable conclusions that flip with small data shifts. Interviewers probe for this by giving mammography- or fraud-style puzzles and watching if you translate to frequencies and integrate prevalence.
Further reading

Probabilistic Machine Learning: An Introduction (Kevin P. Murphy, 2022) — modern, practical textbook covering Bayes’ rule, priors, posteriors, hierarchical models, and decision theory. https://mitpress.mit.edu/9780262046824/probabilistic-machine-learning (mitpress.mit.edu)
Experimentation is a major focus of Data Science across Netflix — industry perspective on Bayesian thinking for A/B decisions and error control (Type-S/M), plus platform context. https://www.engineering.fyi/article/experimentation-is-a-major-focus-of-data-science-across-netflix (engineering.fyi)
Natural frequencies improve Bayesian reasoning (Frontiers in Psychology) — evidence that frequency formats reduce base‑rate neglect; useful for explaining results to stakeholders. https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2015.01473/full (frontiersin.org)

Related concepts