How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Take-home Project rounds at Roblox.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Roblox during technical interviews.

Compute minimum sample size for A/B test | Roblox Interview Question

Quick Overview

Evaluates the ability to perform sample size and power calculations for a two-sample z-test—including estimating outcome standard deviation from historical continuous data—and is categorized under Analytics & Experimentation for a Data Scientist role at an implementation-level applied statistics abstraction.

You are implementing a function to compute the minimum total sample size for an A/B test.

You are given:

observed : a 1D array of historical/baseline metric values (continuous outcome) to estimate the outcome standard deviation.
alpha : significance level for a two-sided test (e.g., 0.05).
power : desired statistical power (e.g., 0.8).
delta : the minimum detectable absolute change in the mean (treatment mean − control mean) you want to be able to detect.

Assumptions:

Two-sample z-test for difference in means.
Treatment and control groups are equal-sized .
Outcome variance is the same in both groups and is estimated from observed .
Use the normal approximation (z critical values).

Task:

Estimate $\sigma$ using the sample standard deviation of observed .
Compute the minimum required per-group sample size $n$ .
Return the minimum total sample size $N=2n$ as an integer, rounded up to the next integer if needed.

Clearly state the formula you use and any edge-case handling (e.g., delta <= 0, sigma == 0).

Quick Overview

You are implementing a function to compute the minimum total sample size for an A/B test.

You are given:

observed : a 1D array of historical/baseline metric values (continuous outcome) to estimate the outcome standard deviation.
alpha : significance level for a two-sided test (e.g., 0.05).
power : desired statistical power (e.g., 0.8).
delta : the minimum detectable absolute change in the mean (treatment mean − control mean) you want to be able to detect.

Assumptions:

Two-sample z-test for difference in means.
Treatment and control groups are equal-sized .
Outcome variance is the same in both groups and is estimated from observed .
Use the normal approximation (z critical values).

Task:

Estimate $\sigma$ using the sample standard deviation of observed .
Compute the minimum required per-group sample size $n$ .
Return the minimum total sample size $N=2n$ as an integer, rounded up to the next integer if needed.

Clearly state the formula you use and any edge-case handling (e.g., delta <= 0, sigma == 0).

Compute minimum sample size for A/B test

Quick Overview

Solution

Comments (0)

Compute minimum sample size for A/B test

Quick Overview

Solution

Comments (0)