Two independent product variants A and B produce a continuous KPI (session revenue in USD) that is right-skewed with ~5% extreme outliers. From historical data you estimate: for A, mean μ_A ≈ 2.30 and sd σ_A ≈ 1.10; for B, mean μ_B ≈ 2.35 and sd σ_B ≈ 1.40. Planned sample sizes are n_A = 8,000 and n_B = 7,500, but you expect 10% missing outcomes in B that are MCAR. Tasks: (1) Decide whether to use a Welch t-test on means, a Mann–Whitney test on distributions, or a trimmed-mean test; justify based on robustness to skew/outliers and on what parameter you want to detect; (2) Compute a 95% CI for (μ_B − μ_A) using Welch’s t (show the Satterthwaite df) and also via a nonparametric bootstrap percentile CI with 10,000 resamples; state pros/cons of each; (3) Approximate the power to detect a true mean lift of +0.05 at α = 0.05 under heteroskedasticity (use Welch variance) and adjust for the 10% MCAR in B; (4) Propose an outlier-robust estimator (e.g., 20% trimmed mean or Huber M-estimator) and compute a 98% CI for the trimmed-mean difference; explain how you would choose the trimming proportion; (5) Suppose you track 20 related KPIs: describe and compute the Benjamini–Hochberg FDR control at q = 0.10 given a sorted p-value list p_(1) ≤ ... ≤ p_(20), and contrast with Bonferroni in terms of power and Type I error; (6) If normality is doubtful, show how to use a variance-stabilizing transform (e.g., Y' = log1p(Y)) and back-transform the CI for an interpretable multiplicative effect, noting the bias correction needed when exponentiating.

This question evaluates a data scientist's competency in robust statistical inference for A/B testing, covering handling of skewed continuous outcomes, extreme outliers, heteroskedasticity, missingness, multiplicity, selection between test statistics, construction of confidence intervals (including bootstrap and transformed CIs), power approximation, and robust estimators like trimmed means or M‑estimators. It is commonly asked in the statistics and experimental-design domain because it probes both conceptual understanding of robustness and multiple-testing principles and practical application skills in selecting appropriate inference methods, computing intervals and power under realistic data issues, and interpreting variance-stabilizing transformations.

How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a hard difficulty Statistics & Math question, commonly asked during Technical Screen rounds at Voleon Group.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Voleon Group during technical interviews.

Compute robust inference under skew and outliers | Voleon Group Interview Question

A/B test with skew, outliers, heteroskedasticity, missingness, and multiplicity

You are comparing two independent product variants that produce a continuous KPI (session revenue in USD). The distribution is right‑skewed with about 5% extreme outliers.

Estimated from historical data:

Variant A: mean μ_A ≈ 2.30, sd σ_A ≈ 1.10
Variant B: mean μ_B ≈ 2.35, sd σ_B ≈ 1.40

Planned sample sizes:

n_A = 8,000
n_B = 7,500, with an expected 10% missing completely at random (MCAR) in B.

Tasks

Test choice: Decide whether to use a Welch t-test on means, a Mann–Whitney test on distributions, or a trimmed-mean test. Justify based on robustness to skew/outliers and on the target parameter.
95% CI for the mean difference: Compute a 95% CI for (μ_B − μ_A) using Welch’s t (show the Satterthwaite df). Also describe how to compute a nonparametric bootstrap percentile 95% CI with 10,000 resamples. State pros/cons of each approach.
Power: Approximate the power to detect a true mean lift of +0.05 at α = 0.05 under heteroskedasticity (use Welch variance) and adjust for the 10% MCAR in B.
Robust estimation: Propose an outlier‑robust estimator (e.g., a 20% trimmed mean or Huber M‑estimator) and compute a 98% CI for the trimmed‑mean difference. Explain how you would choose the trimming proportion.
Multiple testing: Suppose you track 20 related KPIs. Describe and compute the Benjamini–Hochberg FDR control at q = 0.10 given sorted p-values p_(1) ≤ … ≤ p_(20), and contrast with Bonferroni in terms of power and Type I error.
Variance‑stabilizing transform: If normality is doubtful, show how to use Y' = log1p(Y) and back‑transform the CI for an interpretable multiplicative effect, noting the bias correction needed when exponentiating.

A/B test with skew, outliers, heteroskedasticity, missingness, and multiplicity

You are comparing two independent product variants that produce a continuous KPI (session revenue in USD). The distribution is right‑skewed with about 5% extreme outliers.

Estimated from historical data:

Variant A: mean μ_A ≈ 2.30, sd σ_A ≈ 1.10
Variant B: mean μ_B ≈ 2.35, sd σ_B ≈ 1.40

Planned sample sizes:

n_A = 8,000
n_B = 7,500, with an expected 10% missing completely at random (MCAR) in B.

Tasks

Test choice: Decide whether to use a Welch t-test on means, a Mann–Whitney test on distributions, or a trimmed-mean test. Justify based on robustness to skew/outliers and on the target parameter.
95% CI for the mean difference: Compute a 95% CI for (μ_B − μ_A) using Welch’s t (show the Satterthwaite df). Also describe how to compute a nonparametric bootstrap percentile 95% CI with 10,000 resamples. State pros/cons of each approach.
Power: Approximate the power to detect a true mean lift of +0.05 at α = 0.05 under heteroskedasticity (use Welch variance) and adjust for the 10% MCAR in B.
Robust estimation: Propose an outlier‑robust estimator (e.g., a 20% trimmed mean or Huber M‑estimator) and compute a 98% CI for the trimmed‑mean difference. Explain how you would choose the trimming proportion.
Multiple testing: Suppose you track 20 related KPIs. Describe and compute the Benjamini–Hochberg FDR control at q = 0.10 given sorted p-values p_(1) ≤ … ≤ p_(20), and contrast with Bonferroni in terms of power and Type I error.
Variance‑stabilizing transform: If normality is doubtful, show how to use Y' = log1p(Y) and back‑transform the CI for an interpretable multiplicative effect, noting the bias correction needed when exponentiating.

Compute robust inference under skew and outliers

Quick Overview

A/B test with skew, outliers, heteroskedasticity, missingness, and multiplicity

Solution

Comments (0)

Compute robust inference under skew and outliers

Quick Overview

A/B test with skew, outliers, heteroskedasticity, missingness, and multiplicity

Solution

Comments (0)