Evaluate Probability of Positive User Comments and Model Performance
Overview
This question evaluates understanding of probability for independent events and statistical inference for comparing model proportions, measuring competency in modeling i.i.d. Bernoulli outcomes and performing hypothesis tests for model performance in the Statistics & Math domain for Data Scientist roles.
Meta
Aug 4, 2025, 10:55 AM
Data Scientist
Onsite
Statistics & Math
71
0
Social-Media Positivity: Independence and Model Comparison
Context
You are evaluating user comment sentiment and the performance of two models that classify comments as positive. Assume observations are independent and identically distributed (i.i.d.) unless otherwise stated.
Questions
- If the probability that a single user comment is positive is p, what is the probability that two independent comments are both positive?
- Assuming each response is i.i.d., if the first three responses from a user are positive, what is the probability that the fourth response is also positive?
-
Model A returns a positive response in 80% of cases while Model B returns a positive response in 90% of cases. At the 5% significance level, can you conclude Model B is better? Specify:
- The statistical test to use
- Null and alternative hypotheses
- The test statistic (with formula)
- The decision rule at α = 0.05
Note: Treat the 80% and 90% as sample proportions from test sets of sizes n_A and n_B (not provided), and express the decision rule in terms of these sample sizes.
Solution
Login required
Show