How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a hard difficulty Statistics & Math question, commonly asked during Technical Screen rounds at Uber.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Uber during technical interviews.

Compute A/B sample size under clustering

Quick Overview

This question evaluates competency in experimental design and statistical inference for A/B testing, specifically sample size calculation for two-sample proportions with unequal allocation, clustering effects (ICC and design effect), and adjustments for bot removal and data-quality attrition.

A/B Test Sample Size With Unequal Allocation, Clustering, and Attrition

Context

You are planning a two-arm signup A/B test (binary outcome: convert vs. not) with a 2:1 traffic split (control:treatment). You need the analyzable and gross (pre-filter) sample size per variant and the expected test duration, accounting for clustering by user, bot removal, and data-quality attrition.

Given

Baseline conversion (control) p0 = 6% = 0.06
Target relative uplift = +5% ⇒ MDE (absolute) Δ = 0.05 × p0 = 0.003 ⇒ treatment p1 = 0.063
Two-sided α = 0.05, power = 0.80
Allocation: control:treatment = 2:1 ⇒ λ = n_T / n_C = 0.5
Sessions cluster by user: mean sessions/user m = 1.4, ICC ρ = 0.03
15% sessions are bots removed post-hoc
8% expected attrition from data-quality filters
Traffic capacity: 120,000 sessions/day (total)

Task

Provide the formulas for two-sample tests of proportions with unequal allocation.
Compute the required analyzable sample size per variant (control, treatment).
Apply the design effect for clustering.
Inflate to gross (pre-filter) sessions accounting for 15% bots and 8% attrition.
Convert the result to an estimated test duration given 120,000 sessions/day under a 2:1 split.
State assumptions and how violations (variance mis-specification, sequential peeking) would change the plan.

Quick Overview

Context

Given

Baseline conversion (control) p0 = 6% = 0.06

Target relative uplift = +5% ⇒ MDE (absolute) Δ = 0.05 × p0 = 0.003 ⇒ treatment p1 = 0.063

Two-sided α = 0.05, power = 0.80

Allocation: control:treatment = 2:1 ⇒ λ = n_T / n_C = 0.5

Sessions cluster by user: mean sessions/user m = 1.4, ICC ρ = 0.03

15% sessions are bots removed post-hoc

8% expected attrition from data-quality filters

Traffic capacity: 120,000 sessions/day (total)

Task

Provide the formulas for two-sample tests of proportions with unequal allocation.

Compute the required analyzable sample size per variant (control, treatment).

Apply the design effect for clustering.

Inflate to gross (pre-filter) sessions accounting for 15% bots and 8% attrition.

Convert the result to an estimated test duration given 120,000 sessions/day under a 2:1 split.

State assumptions and how violations (variance mis-specification, sequential peeking) would change the plan.

Compute A/B sample size under clustering

Quick Overview

A/B Test Sample Size With Unequal Allocation, Clustering, and Attrition

Context

Given

Task

Solution

Comments (0)

Compute A/B sample size under clustering

Quick Overview

A/B Test Sample Size With Unequal Allocation, Clustering, and Attrition

Context

Given

Task

Solution

Comments (0)