Compute sample size and significance

Q: Compute sample size and significance

This is a Statistics & Math interview question from Flatiron Health for Data Scientist roles. View the full question and solution on PracHub.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Question

You are planning a two-variant A/B test with equal allocation and a binary primary metric (conversion). Baseline rate p0 = 0.045. You want to detect a +10% relative lift (MDE) vs baseline at two-sided alpha = 0.05 and power = 0.80. Assume independent Bernoulli trials, no continuity correction. A) Derive the per-variant sample size using the standard normal-approximation formula for two-proportion tests. Show all intermediate values (z-scores, pooled variance term) and the final per-arm n. B) If you apply CUPED that reduces variance by 30%, recompute the per-variant sample size and quantify the absolute and relative reduction vs part A. C) Mid-experiment, you observe pA = 0.045 and pB = 0.052 with nA = nB = 50,000 users collected without peeking. Perform a two-sided z-test for pB - pA, report the z, p-value, and a 95% CI for the difference in percentage points. State whether it’s significant at alpha = 0.05 and interpret practically.

Compute sample size and significance

Comments (0)