CLT, Confidence Intervals, and Hypothesis Tests

What it is The central limit theorem (CLT) explains why averages from many independent, similar observations are approximately normal, enabling simple uncertainty calculations. Confidence intervals quantify the plausible range for a population quantity, and hypothesis tests assess whether observed effects are likely under a specified null model.

Why interviewers ask about it Product data scientists design and interpret experiments that decide if features (e.g., feed ranking tweaks, notification rules) should ship. You need to size tests, compute intervals around lifts, and explain p-values, error rates, and assumptions so launches are safe and defensible.

Core ideas to know

A common pitfall Candidates often say “there’s a 95% chance the true mean is in my 95% interval,” which confuses frequentist coverage with probability about a fixed parameter. Another frequent mistake is peeking—checking significance repeatedly without correction—which inflates false positives and ships bad variants. Many also ignore bucketing or event logging issues that break independence (e.g., sample ratio mismatch), making p-values and intervals meaningless. Strong answers state assumptions, check data quality, and justify the test plan (fixed-horizon or sequential) up front.

Further reading