Compute minimum sample size for A/B test
Company: Roblox
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Take-home Project
You are implementing a function to compute the **minimum total sample size** for an A/B test.
You are given:
- `observed`: a 1D array of historical/baseline metric values (continuous outcome) to estimate the outcome standard deviation.
- `alpha`: significance level for a **two-sided** test (e.g., 0.05).
- `power`: desired statistical power (e.g., 0.8).
- `delta`: the minimum detectable absolute change in the mean (treatment mean − control mean) you want to be able to detect.
Assumptions:
- Two-sample **z-test** for difference in means.
- Treatment and control groups are **equal-sized**.
- Outcome variance is the same in both groups and is estimated from `observed`.
- Use the normal approximation (z critical values).
Task:
1) Estimate \(\sigma\) using the sample standard deviation of `observed`.
2) Compute the minimum required per-group sample size \(n\).
3) Return the **minimum total sample size** \(N=2n\) as an integer, **rounded up** to the next integer if needed.
Clearly state the formula you use and any edge-case handling (e.g., `delta <= 0`, `sigma == 0`).
Quick Answer: Evaluates the ability to perform sample size and power calculations for a two-sample z-test—including estimating outcome standard deviation from historical continuous data—and is categorized under Analytics & Experimentation for a Data Scientist role at an implementation-level applied statistics abstraction.