How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a medium difficulty Statistics & Math question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Explain Statistical Concepts in A/B Testing and Corrections

Quick Overview

This interview question evaluates statistical assumptions, formulas, estimation strategy, uncertainty, edge cases, and interpretation in a realistic interview setting. A strong answer for Explain Statistical Concepts in A/B Testing and Corrections states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Explain Statistical Concepts in A/B Testing and Corrections

A/B Testing: p-values, Power, and Error Rates with Multiple Comparisons

Context

You are reviewing the results of an online A/B experiment. Stakeholders question whether your findings are statistically valid, especially because you track several metrics and may have more than two variants.

Task

Define the following in the context of A/B testing:
- p-value
- Type I error
- Type II error
- Statistical power
Explain why tracking multiple metrics and/or testing multiple variants inflates false positives and requires corrections (e.g., Bonferroni).
Demonstrate with a concrete numerical example how family-wise error rate (FWER) grows with the number of tests and how Bonferroni controls it.

Constraints & Assumptions

Preserve the scope, facts, inputs, and requested outputs from the prompt above.
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

Clarify the random variables, distributional assumptions, independence assumptions, and desired output.
Show enough derivation for the interviewer to follow the reasoning.
Explain how you would validate the result with simulation or sensitivity checks.

What a Strong Answer Covers Guidance

A correct setup with definitions, formulas, and boundary conditions.
A step-by-step derivation or estimation plan.
Interpretation of the result, including uncertainty and practical limitations.
Checks for assumptions, edge cases, and numerical stability.

Follow-up Questions Guidance

How would the result change if the assumptions were relaxed?
Can you verify the answer with a simulation?
What is the most likely source of estimation error?

Quick Overview

Task

Define the following in the context of A/B testing:

p-value
Type I error
Type II error
Statistical power

Explain why tracking multiple metrics and/or testing multiple variants inflates false positives and requires corrections (e.g., Bonferroni).

Demonstrate with a concrete numerical example how family-wise error rate (FWER) grows with the number of tests and how Bonferroni controls it.

Constraints & Assumptions

Preserve the scope, facts, inputs, and requested outputs from the prompt above.

If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.

Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

Clarify the random variables, distributional assumptions, independence assumptions, and desired output.

Show enough derivation for the interviewer to follow the reasoning.

Explain how you would validate the result with simulation or sensitivity checks.

What a Strong Answer Covers Guidance

A correct setup with definitions, formulas, and boundary conditions.

A step-by-step derivation or estimation plan.

Interpretation of the result, including uncertainty and practical limitations.

Checks for assumptions, edge cases, and numerical stability.

Follow-up Questions Guidance

How would the result change if the assumptions were relaxed?

Can you verify the answer with a simulation?

What is the most likely source of estimation error?

Explain Statistical Concepts in A/B Testing and Corrections

Quick Overview

Explain Statistical Concepts in A/B Testing and Corrections

Explain Statistical Concepts in A/B Testing and Corrections

A/B Testing: p-values, Power, and Error Rates with Multiple Comparisons

Context

Task

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer

Explain Statistical Concepts in A/B Testing and Corrections

Quick Overview

Explain Statistical Concepts in A/B Testing and Corrections

Explain Statistical Concepts in A/B Testing and Corrections

A/B Testing: p-values, Power, and Error Rates with Multiple Comparisons

Context

Task

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer