Probability, Conditional Expectation And Bayes Rule

What's being tested

Interviewers probe your ability to reason about uncertainty: forming and manipulating conditional probabilities, computing conditional expectations, and applying Bayes' rule to update beliefs from data. For a Data Scientist, this shows you can interpret noisy signals (A/B results, classifier outputs, diagnostic tests), combine prior knowledge with new evidence, and make decisions that balance error types and costs. Interviewers also check for numerical and conceptual traps — base-rate effects, dependence assumptions, and when to use frequentist vs Bayesian summaries.

Core knowledge

Conditional probability: $P(A\mid B)=\frac{P(A\cap B)}{P(B)}$ and requires explicit conditioning set; always check denominator nonzero and interpret conditioning as “given information”.
Bayes' rule: $P(\theta\mid D)=\frac{P(D\mid\theta)P(\theta)}{P(D)}$ where $P(D)=\sum_{\theta}P(D\mid\theta)P(\theta)$ (discrete) — the basis for updating priors to posteriors and for diagnostic-test PPV/NPV.
Law of total probability / expectation: $P(A)=\sum_i P(A\mid B_i)P(B_i),\quad E[X]=E[E[X\mid Y]]$ use to marginalize latent variables and to compute predictive distributions.
Conditional expectation properties: $E[aX+b\mid Y]=aE[X\mid Y]+b$ ; $E[1_{A}\mid Y]=P(A\mid Y)$ — use indicator trick to convert probabilities to expectations.
Indicator & covariance relations: $Cov(X,Y)=E[XY]-E[X]E[Y]$ ; conditional independence matters: $X\perp\!\!\!\perp Y\mid Z$ simplifies updates and factorizes likelihoods.
Bayesian conjugacy: Beta-Binomial: prior Beta $(\alpha,\beta)$ , data $k$ successes/ $n$ trials → posterior Beta $(\alpha+k,\beta+n-k)$ ; Gaussian conjugacy when variance known. Use for analytic posteriors and fast updates.
Predictive probabilities & PPV: For tests with sensitivity $s$ and specificity $t$ and prevalence $\pi$ , positive predictive value $\text{PPV}=\frac{s\pi}{s\pi+(1-t)(1-\pi)}$ — highlights base-rate impact on post-test probability.
Log-odds updates: Bayes in log space: log-odds posterior = log-odds prior + log-likelihood-ratio; stable numerics and intuitive additive updates.
Decision thresholds and expected loss: Choose action by minimizing expected loss under posterior; threshold often compares posterior odds to cost ratio (false positive vs false negative).
Empirical Bayes & hierarchical models: Estimate priors from pooled data for shrinkage; useful when many similar cohorts have sparse data.
Numerical practice: compute probabilities in log-space to avoid underflow, run small-sample sensitivity to priors, use Monte Carlo (draws) for integrals like $P(\theta_1>\theta_2)$ when closed-form is hard.
Common discrete vs continuous nuance: probability densities require Jacobians when changing variables; avoid treating pdf values as probabilities without integration over intervals.

Worked example — diagnostic test / positive predictive value

Framing (first 30s): ask for the prevalence (prior), the test sensitivity and specificity (likelihood), and what decision follows from a positive result (treatment cost vs harm). Skeleton of response: (1) write Bayes' rule for P(disease|positive), (2) plug numbers to compute PPV, (3) interpret result and perform sensitivity analysis across plausible prevalence values. A strong candidate shows the formula $\text{PPV}=\frac{\text{sensitivity}\times\pi}{\text{sensitivity}\times\pi+(1-\text{specificity})\times(1-\pi)}$ and computes a numeric example. Tradeoff to flag: for rare diseases, even high sensitivity/specificity can yield low PPV — thus specificity matters more to avoid false positives. Close by recommending actions: confirmatory testing, estimate population prevalence, or incorporate costs via expected-loss threshold; if more time, propose a hierarchical model to pool prevalence estimates across subpopulations.

A second angle — A/B test with Beta priors

Framing differs: here the parameter is a click-through probability for control and treatment. Use a Beta $(\alpha,\beta)$ prior for each variant, observe successes/failures, update to Beta posteriors. Main pillars: (1) analytic posterior via conjugacy, (2) compute $P(p_{A}<p_{B})$ via Monte Carlo sampling from posteriors or closed-form Beta-Beta integrals, (3) decide using an expected-loss metric (e.g., lift times monetization). Tradeoffs: choice of prior affects posterior for small samples — explicit sensitivity analysis or empirical Bayes can defend your prior. For large- $N$ , normal approximations and frequentist p-values converge but Bayesian posterior gives richer probability statements about which variant is better.

Common pitfalls

Pitfall: confusing $P(A|B)$ with $P(B|A)$ .
Many candidates mechanically swap these; always write Bayes' rule and check which quantity is prior and which is likelihood. In diagnostics and classifiers, this error produces grossly wrong posterior interpretation.

Pitfall: ignoring base rates (prevalence).
Arguing “test is 99% accurate so positive means disease” without computing PPV misses that low prevalence collapses post-test probability — explain sensitivity of posterior to prior.

Pitfall: reporting point estimates without decision context.
Giving only a posterior mean or p-value misses actions; always map probabilities to decisions via expected loss or business metric (cost of false positive vs false negative).

Connections

These ideas connect naturally to causal inference (conditioning vs do-operations), hypothesis testing / sequential testing (stopping rules and Type I/II tradeoffs), and Bayesian hierarchical modeling for pooling sparse subgroup estimates. Interviewers may pivot to classifier calibration, A/B test ramping, or multi-armed bandit decision-making.