How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a medium difficulty Statistics & Math question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Determine Superiority of Model A Using Hypothesis Testing

Last updated: Mar 29, 2026

Quick Overview

This question evaluates competency in statistical hypothesis testing for comparing proportions — including selection of an appropriate two-proportion test, computation of pooled and unpooled standard errors, z-statistics and p-values, construction of confidence intervals, and considerations of statistical power and sample-size requirements.

Scenario

A search feature marks a user session as a success only if both the relevancy and accuracy binary flags equal 1.

Two ranking models were A/B tested independently with equal traffic:

Model A: 100 users, 90 successes
Model B: 100 users, 85 successes

Task

Using only these data, determine whether you can conclude that Model A is better than Model B at a 5% significance level.

Provide:

The appropriate statistical test and hypotheses.
The pooled variance, standard error, z-statistic, and p-value.
A 95% confidence interval for the difference in success rates.
An interpretation (statistical and practical significance).
A brief comment on statistical power and the sample size needed to detect a 5 percentage-point lift.

Hints

Use a two-proportion z-test (pooled standard error for the hypothesis test).
For the 95% CI, use the unpooled standard error.
Comment on power: whether n=100 per arm is enough to detect a 5 pp difference.

Solution

Show

Submit Your Answer to Earn 20XP

Loading comments...

Browse More Questions

More Statistics & Math•More Meta•More Data Scientist•Meta Data Scientist•Meta Statistics & Math•Data Scientist Statistics & Math