Compute an A/B test p-value by hand

Quick Overview

This question evaluates understanding of hypothesis testing for proportions, specifically the two-proportion z-test, p-value computation and interpretation of statistical significance in an A/B test with binary outcomes.

Compute an A/B test p-value by hand

Company: Amazon

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Technical Screen

In an A/B test on a game feature, you measure **conversion rate** (binary outcome). - Control: n₁ = 1000 users, x₁ = 120 conversions - Treatment: n₂ = 980 users, x₂ = 145 conversions Tasks: 1) Compute the **two-sided p-value** for the null hypothesis H₀: p₁ = p₂ using a **two-proportion z-test** (show formulas and intermediate values). 2) State whether the result is significant at α = 0.05. 3) (Implementation) Describe how you would compute this p-value in Python **without using SciPy** (you may use `math` only).

Quick Answer: This question evaluates understanding of hypothesis testing for proportions, specifically the two-proportion z-test, p-value computation and interpretation of statistical significance in an A/B test with binary outcomes.

|Home/Statistics & Math/Amazon
Amazon logo
Amazon
Nov 4, 2025, 12:00 AM
mediumData ScientistTechnical ScreenStatistics & Math
12
0

In an A/B test on a game feature, you measure conversion rate (binary outcome).

  • Control: n₁ = 1000 users, x₁ = 120 conversions
  • Treatment: n₂ = 980 users, x₂ = 145 conversions

Tasks:

  1. Compute the two-sided p-value for the null hypothesis H₀: p₁ = p₂ using a two-proportion z-test (show formulas and intermediate values).
  2. State whether the result is significant at α = 0.05.
  3. (Implementation) Describe how you would compute this p-value in Python without using SciPy (you may use math only).
Loading comments...