Compute an A/B test p-value by hand

Q: Compute an A/B test p-value by hand

This question evaluates understanding of hypothesis testing for proportions, specifically the two-proportion z-test, p-value computation and interpretation of statistical significance in an A/B test with binary outcomes.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Question

In an A/B test on a game feature, you measure conversion rate (binary outcome).

Control: n₁ = 1000 users, x₁ = 120 conversions
Treatment: n₂ = 980 users, x₂ = 145 conversions

Tasks:

Compute the two-sided p-value for the null hypothesis H₀: p₁ = p₂ using a two-proportion z-test (show formulas and intermediate values).
State whether the result is significant at α = 0.05.
(Implementation) Describe how you would compute this p-value in Python without using SciPy (you may use math only).

Compute an A/B test p-value by hand

Quick Overview

Solution

Comments (0)