PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Statistics & Math/Uber

Design an A/B test; choose Z vs T

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a candidate's competency in experimental design and statistical inference, covering sample-size and power calculations, the choice between Z and T tests, sequential testing/alpha-spending strategies, and causal-inference planning for non-randomized rollouts.

  • hard
  • Uber
  • Statistics & Math
  • Data Scientist

Design an A/B test; choose Z vs T

Company: Uber

Role: Data Scientist

Category: Statistics & Math

Difficulty: hard

Interview Round: Technical Screen

You are planning an A/B test on a signup funnel. Baseline conversion p0 = 0.050, target minimum detectable effect (MDE) = +10% relative (p1 = 0.055). Daily eligible traffic = 50,000 visitors, but only 60% meet targeting criteria and 90% of eligible visitors are successfully bucketed. 1:1 split, two-sided alpha = 0.05, power = 0.80. Assume independence across users. (a) Compute the required sample size per variant and the expected calendar duration (in days) to reach it given the traffic constraints above. State your formula and any continuity or pooled-variance assumptions. (b) Explain whether you would use a Z-test or a T-test for the primary proportion metric and why. Under what conditions do their results materially differ? Include how estimating variance from data, small-sample corrections, and unequal sample sizes/variances affect your choice. (c) If product wants the test to stop “as soon as it looks good,” propose a sequential testing or alpha-spending approach (e.g., group-sequential boundaries) that controls Type I error. Specify stopping rules and how they change the nominal sample size and expected duration. (d) Suppose randomization is not possible for a related rollout. Sketch a causal inference plan: draw a DAG to identify confounders, propose an identification strategy (e.g., difference-in-differences with parallel trends checks or an instrumental variable) and list key assumptions you would need to defend. How would you validate those assumptions in practice?

Quick Answer: This question evaluates a candidate's competency in experimental design and statistical inference, covering sample-size and power calculations, the choice between Z and T tests, sequential testing/alpha-spending strategies, and causal-inference planning for non-randomized rollouts.

Related Interview Questions

  • Should Uber double member discounts? - Uber (medium)
  • Compare Two Coin Proportions - Uber (medium)
  • Analyze the Accident-Rate Spike - Uber (easy)
  • How do you derive CDF from a PDF? - Uber (easy)
  • Derive a CDF from a PDF - Uber (medium)
Uber logo
Uber
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Statistics & Math
5
0

A/B Test on a Signup Funnel: Sample Size, Test Choice, Sequential Design, and Causal Plan

Context

You are planning a two-variant A/B test on a signup funnel with the following parameters:

  • Baseline conversion p0 = 0.050
  • Target minimum detectable effect (MDE) = +10% relative → p1 = 0.055 (Δ = 0.005 absolute)
  • Two-sided alpha = 0.05; power = 0.80
  • Allocation = 1:1
  • Daily site traffic = 50,000 visitors; only 60% meet targeting criteria; 90% of those are successfully bucketed
  • Assume independence across users

Tasks

(a) Compute the required sample size per variant and the expected calendar duration (in days) to reach it given the traffic constraints above. State your formula and any continuity or pooled-variance assumptions.

(b) Explain whether you would use a Z-test or a T-test for the primary proportion metric and why. Under what conditions do their results materially differ? Include how estimating variance from data, small-sample corrections, and unequal sample sizes/variances affect your choice.

(c) If product wants the test to stop “as soon as it looks good,” propose a sequential testing or alpha-spending approach (e.g., group-sequential boundaries) that controls Type I error. Specify stopping rules and how they change the nominal sample size and expected duration.

(d) Suppose randomization is not possible for a related rollout. Sketch a causal inference plan: draw a DAG to identify confounders, propose an identification strategy (e.g., difference-in-differences with parallel trends checks or an instrumental variable) and list key assumptions you would need to defend. How would you validate those assumptions in practice?

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Statistics & Math•More Uber•More Data Scientist•Uber Data Scientist•Uber Statistics & Math•Data Scientist Statistics & Math
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.