Evaluate Probability of Positive User Comments and Model Performance
Quick Overview
Evaluate Probability of Positive User Comments and Model Performance evaluates statistical assumptions, formulas, estimation strategy, uncertainty, edge cases, and interpretation in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Evaluate Probability of Positive User Comments and Model Performance
Company: Meta
Role: Data Scientist
Category: Statistics & Math
Difficulty: medium
Interview Round: Onsite
##### Scenario
You are an analyst evaluating user comments and model performance on a social-media platform.
##### Question
If the probability that a single user comment is positive is p, what is the probability that two independent comments are both positive? Assuming each response is i.i.d., if the first three responses from a user are positive, what is the probability that the fourth response is also positive? Model A returns a positive response in 80 % of cases while Model B returns a positive response in 90 % of cases. At the 5 % significance level, can you conclude Model B is better? Detail the statistical test, null/alternative hypotheses, test statistic and decision rule.
##### Hints
Use the multiplication rule for independent events and a two-proportion z-test with a pooled estimate of p.
Quick Answer: Evaluate Probability of Positive User Comments and Model Performance evaluates statistical assumptions, formulas, estimation strategy, uncertainty, edge cases, and interpretation in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
Evaluate Probability of Positive User Comments and Model Performance
Meta
Aug 4, 2025, 10:55 AM
mediumData ScientistOnsiteStatistics & Math
107
0
Evaluate Probability of Positive User Comments and Model Performance
Social-Media Positivity: Independence and Model Comparison
Context
You are evaluating user comment sentiment and the performance of two models that classify comments as positive. Assume observations are independent and identically distributed (i.i.d.) unless otherwise stated.
Questions
If the probability that a single user comment is positive is p, what is the probability that two independent comments are both positive?
Assuming each response is i.i.d., if the first three responses from a user are positive, what is the probability that the fourth response is also positive?
Model A returns a positive response in 80% of cases while Model B returns a positive response in 90% of cases. At the 5% significance level, can you conclude Model B is better? Specify:
The statistical test to use
Null and alternative hypotheses
The test statistic (with formula)
The decision rule at α = 0.05
Note: Treat the 80% and 90% as sample proportions from test sets of sizes n_A and n_B (not provided), and express the decision rule in terms of these sample sizes.
Constraints & Assumptions
Preserve the scope, facts, inputs, and requested outputs from the prompt above.
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.
Clarifying Questions to Ask Guidance
Clarify the random variables, distributional assumptions, independence assumptions, and desired output.
Show enough derivation for the interviewer to follow the reasoning.
Explain how you would validate the result with simulation or sensitivity checks.
What a Strong Answer Covers Guidance
A correct setup with definitions, formulas, and boundary conditions.
A step-by-step derivation or estimation plan.
Interpretation of the result, including uncertainty and practical limitations.
Checks for assumptions, edge cases, and numerical stability.
Follow-up Questions Guidance
How would the result change if the assumptions were relaxed?
Can you verify the answer with a simulation?
What is the most likely source of estimation error?