Compute minimum sample size for A/B test

Q: Compute minimum sample size for A/B test

Evaluates the ability to perform sample size and power calculations for a two-sample z-test—including estimating outcome standard deviation from historical continuous data—and is categorized under Analytics & Experimentation for a Data Scientist role at an implementation-level applied statistics abstraction.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

You are implementing a function to compute the minimum total sample size for an A/B test.

You are given:

observed : a 1D array of historical/baseline metric values (continuous outcome) to estimate the outcome standard deviation.
alpha : significance level for a two-sided test (e.g., 0.05).
power : desired statistical power (e.g., 0.8).
delta : the minimum detectable absolute change in the mean (treatment mean − control mean) you want to be able to detect.

Assumptions:

Two-sample z-test for difference in means.
Treatment and control groups are equal-sized .
Outcome variance is the same in both groups and is estimated from observed .
Use the normal approximation (z critical values).

Task:

Estimate $\sigma$ using the sample standard deviation of observed .
Compute the minimum required per-group sample size $n$ .
Return the minimum total sample size $N=2n$ as an integer, rounded up to the next integer if needed.

Clearly state the formula you use and any edge-case handling (e.g., delta <= 0, sigma == 0).

Compute minimum sample size for A/B test

Quick Overview

Solution

Comments (0)