How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a medium difficulty Statistics & Math question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Explain Statistical Concepts in A/B Testing and Corrections

Last updated: Mar 29, 2026

Quick Overview

This question evaluates understanding of statistical inference in A/B testing—specifically familiarity with p-values, Type I and II errors, statistical power, and multiple-comparison corrections—and the competency to assess experiment validity and control false-positive rates.

A/B Testing: p-values, Power, and Error Rates with Multiple Comparisons

Context

You are reviewing the results of an online A/B experiment. Stakeholders question whether your findings are statistically valid, especially because you track several metrics and may have more than two variants.

Task

Define the following in the context of A/B testing:
- p-value
- Type I error
- Type II error
- Statistical power
Explain why tracking multiple metrics and/or testing multiple variants inflates false positives and requires corrections (e.g., Bonferroni).
Demonstrate with a concrete numerical example how family-wise error rate (FWER) grows with the number of tests and how Bonferroni controls it.

Solution

Show

Submit Your Answer

Loading comments...

Browse More Questions

More Statistics & Math•More Meta•More Data Scientist•Meta Data Scientist•Meta Statistics & Math•Data Scientist Statistics & Math