PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/LinkedIn

Decide best email variant using stratified A/B analysis

Last updated: Mar 29, 2026

Quick Overview

This question evaluates statistical inference and experimentation competencies, focusing on stratified A/B testing, estimation and interpretation of treatment effects (conversion rates, odds ratios, and confidence intervals), and assessment of effect heterogeneity between strata.

  • medium
  • LinkedIn
  • Analytics & Experimentation
  • Data Scientist

Decide best email variant using stratified A/B analysis

Company: LinkedIn

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Technical Screen

You ran an email A/B test across two strata (week/location). Week 1 (Los Angeles): Email A sent 100,000 with 10,000 responses; Email B sent 10,000 with 1,500 responses. Week 2 (New York): Email A sent 10,000 with 400 responses; Email B sent 100,000 with 6,000 responses. Decide which variant is better. Requirements: (1) Compute stratum-specific conversion rates and 95% CIs; (2) Test for a common treatment effect across strata using a Mantel–Haenszel estimate (report the common odds ratio and 95% CI) and state the two-sided p-value (alpha = 0.05); (3) Test for effect heterogeneity (e.g., Breslow–Day or an equivalent interaction test) and interpret; (4) Compute the naive pooled difference if you ignore stratification, explain whether Simpson’s paradox occurs here, and why; (5) Make a recommendation (A or B) with justification that reconciles the stratified and naive views. Clearly state any assumptions (e.g., independence, single exposure per user).

Quick Answer: This question evaluates statistical inference and experimentation competencies, focusing on stratified A/B testing, estimation and interpretation of treatment effects (conversion rates, odds ratios, and confidence intervals), and assessment of effect heterogeneity between strata.

Related Interview Questions

  • Test whether US uploads more videos - LinkedIn (easy)
  • Resolve Simpson’s paradox in email A/B test - LinkedIn (easy)
  • Choose single queue vs multiple queues - LinkedIn (easy)
  • Resolve Simpson’s paradox in A/B email test - LinkedIn (easy)
  • Do US members upload more videos than non-US? - LinkedIn (easy)
LinkedIn logo
LinkedIn
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
4
0

Stratified A/B Test Across Two Strata (Week/Location)

You ran an email A/B test across two strata defined by week/location. Each user receives at most one email and either responds (success) or not (failure).

Strata and counts:

  • Stratum 1 — Week 1 (Los Angeles):
    • A: 100,000 sent, 10,000 responses
    • B: 10,000 sent, 1,500 responses
  • Stratum 2 — Week 2 (New York):
    • A: 10,000 sent, 400 responses
    • B: 100,000 sent, 6,000 responses

Tasks:

  1. Compute stratum-specific conversion rates (A and B) and 95% confidence intervals.
  2. Using a Mantel–Haenszel (MH) approach, estimate the common treatment effect across strata: report the common odds ratio (OR) with 95% CI and a two-sided p-value (alpha = 0.05).
  3. Test for effect heterogeneity across strata (e.g., Breslow–Day or an equivalent interaction test) and interpret.
  4. Compute the naive pooled difference in conversion (ignoring stratification) and state whether Simpson’s paradox occurs, and why.
  5. Recommend A or B, reconciling the stratified and naive results.

Assume standard A/B test conditions unless stated otherwise (e.g., independent Bernoulli outcomes, single exposure per user, no interference, and randomization within each stratum).

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More LinkedIn•More Data Scientist•LinkedIn Data Scientist•LinkedIn Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.