PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Statistics & Math/Flatiron Health

Compute sample size and significance

Last updated: Mar 29, 2026

Quick Overview

This question evaluates understanding of statistical power, two-proportion sample-size calculation, hypothesis testing, and variance-reduction techniques (CUPED) in the context of binary A/B experiments.

  • Medium
  • Flatiron Health
  • Statistics & Math
  • Data Scientist

Compute sample size and significance

Company: Flatiron Health

Role: Data Scientist

Category: Statistics & Math

Difficulty: Medium

Interview Round: Technical Screen

You are planning a two-variant A/B test with equal allocation and a binary primary metric (conversion). Baseline rate p0 = 0.045. You want to detect a +10% relative lift (MDE) vs baseline at two-sided alpha = 0.05 and power = 0.80. Assume independent Bernoulli trials, no continuity correction. A) Derive the per-variant sample size using the standard normal-approximation formula for two-proportion tests. Show all intermediate values (z-scores, pooled variance term) and the final per-arm n. B) If you apply CUPED that reduces variance by 30%, recompute the per-variant sample size and quantify the absolute and relative reduction vs part A. C) Mid-experiment, you observe pA = 0.045 and pB = 0.052 with nA = nB = 50,000 users collected without peeking. Perform a two-sided z-test for pB - pA, report the z, p-value, and a 95% CI for the difference in percentage points. State whether it’s significant at alpha = 0.05 and interpret practically.

Quick Answer: This question evaluates understanding of statistical power, two-proportion sample-size calculation, hypothesis testing, and variance-reduction techniques (CUPED) in the context of binary A/B experiments.

Flatiron Health logo
Flatiron Health
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Statistics & Math
2
0

You are planning a two-variant A/B test with equal allocation and a binary primary metric (conversion). Baseline rate p0 = 0.045. You want to detect a +10% relative lift (MDE) vs baseline at two-sided alpha = 0.05 and power = 0.80. Assume independent Bernoulli trials, no continuity correction. A) Derive the per-variant sample size using the standard normal-approximation formula for two-proportion tests. Show all intermediate values (z-scores, pooled variance term) and the final per-arm n. B) If you apply CUPED that reduces variance by 30%, recompute the per-variant sample size and quantify the absolute and relative reduction vs part A. C) Mid-experiment, you observe pA = 0.045 and pB = 0.052 with nA = nB = 50,000 users collected without peeking. Perform a two-sided z-test for pB - pA, report the z, p-value, and a 95% CI for the difference in percentage points. State whether it’s significant at alpha = 0.05 and interpret practically.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Statistics & Math•More Flatiron Health•More Data Scientist•Flatiron Health Data Scientist•Flatiron Health Statistics & Math•Data Scientist Statistics & Math
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.