Compute A/B test sample size and Bayes posterior

Q: Compute A/B test sample size and Bayes posterior

This question evaluates understanding of statistical power and sample size calculation for a two-sided A/B z-test as well as basic Bayesian posterior computation, with emphasis on estimating outcome variance and updating probabilities in the Statistics & Math domain.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Question

Loading...

Part A: Minimum sample size for a two-sided A/B z-test

You are given:

observed : an array of numeric outcomes from historical data (use it to estimate the outcome standard deviation)
alpha : significance level for a two-sided z-test (e.g., 0.05)
power : desired power (e.g., 0.8)
delta : the minimum detectable absolute difference in means between treatment and control

Assume:

Treatment and control groups are equal size .
The outcome variance is the same in both groups.
You may approximate using a normal (z) test and use sigma = std(observed) as the standard deviation estimate.

Task: compute the minimum total sample size N_total = N_control + N_treatment required to detect a mean difference of delta with significance alpha and power power. Round up to the nearest integer.

Part B: One-step Bayes’ rule

Given probabilities:

p_A = P(A)
p_B_given_A = P(B|A)
p_B_given_notA = P(B|¬A)

Task: compute and return P(A|B).

Output

Return:

N_total (integer)
p_A_given_B (float)

Compute A/B test sample size and Bayes posterior

Part A: Minimum sample size for a two-sided A/B z-test

Part B: One-step Bayes’ rule

Output

Solution

Comments (0)

Compute A/B test sample size and Bayes posterior

Overview

Part A: Minimum sample size for a two-sided A/B z-test

Part B: One-step Bayes’ rule

Output

Solution

Comments (0)