PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Statistics & Math/Meta

Test two models' proportions for significance

Last updated: Mar 29, 2026

Quick Overview

This question evaluates proficiency in statistical inference for proportions, covering hypothesis testing, confidence intervals, power and sample-size calculations, multiple-testing correction, and comparison of frequentist versus Bayesian approaches within the Statistics & Math domain for data scientist roles.

  • Medium
  • Meta
  • Statistics & Math
  • Data Scientist

Test two models' proportions for significance

Company: Meta

Role: Data Scientist

Category: Statistics & Math

Difficulty: Medium

Interview Round: Onsite

Two search models, A and B, were each used once by 100 distinct users (one query per user). Success is defined per query by your composite metric (success=1, failure=0). Model A had 90 successes, Model B had 85. Using a two-sided test at alpha=0.05: 1) State H0 and H1, choose the appropriate test (pooled two-proportion z-test), compute the test statistic and p-value, and conclude whether A outperforms B. 2) Compute a 95% confidence interval for pA−pB and interpret it for practical significance. 3) What per-arm sample size is needed to detect a +5 percentage-point uplift (baseline 85%) with 80% power at alpha=0.05? Show formulas/inputs. 4) If you simultaneously test these two models across 10 independent intents, apply a Bonferroni correction and say whether your conclusion changes. 5) Briefly explain when you would prefer an exact test or a Bayesian comparison and what you would report in each case.

Quick Answer: This question evaluates proficiency in statistical inference for proportions, covering hypothesis testing, confidence intervals, power and sample-size calculations, multiple-testing correction, and comparison of frequentist versus Bayesian approaches within the Statistics & Math domain for data scientist roles.

Related Interview Questions

  • Compute probability an account is fake - Meta (easy)
  • Compute Bayes probability for fake accounts - Meta (easy)
  • Compute probabilities for chatbot response quality - Meta (easy)
  • Compute posterior fake probability using Bayes' rule - Meta (medium)
  • Estimate bots and CI from DAU spike - Meta (medium)
Meta logo
Meta
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Statistics & Math
3
0

Two search models, A and B, were each used once by 100 distinct users (one query per user). Success is defined per query by your composite metric (success=1, failure=0). Model A had 90 successes, Model B had 85. Using a two-sided test at alpha=0.05: 1) State H0 and H1, choose the appropriate test (pooled two-proportion z-test), compute the test statistic and p-value, and conclude whether A outperforms B. 2) Compute a 95% confidence interval for pA−pB and interpret it for practical significance. 3) What per-arm sample size is needed to detect a +5 percentage-point uplift (baseline 85%) with 80% power at alpha=0.05? Show formulas/inputs. 4) If you simultaneously test these two models across 10 independent intents, apply a Bonferroni correction and say whether your conclusion changes. 5) Briefly explain when you would prefer an exact test or a Bayesian comparison and what you would report in each case.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Statistics & Math•More Meta•More Data Scientist•Meta Data Scientist•Meta Statistics & Math•Data Scientist Statistics & Math
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.