Compute p-values and adjust multiplicity
Role: Data Scientist
Category: Statistics & Math
Difficulty: medium
Interview Round: Onsite
You test if a coin is fair after 20 flips and observe 17 heads. (a) Compute the exact two-sided p-value under a Binomial(20, 0.5) model. (b) Approximate the p-value using a normal z-test with and without continuity correction; discuss accuracy. (c) Now you simultaneously test 6 independent features of a product using similar binomial tests at familywise α=0.05; apply Bonferroni and Holm corrections and state your decision rule. (d) Explain precisely why a p-value is not the probability that H0 is true. (e) When should you use z-test vs t-test for a mean difference, and what assumptions must hold?
Quick Answer: This question evaluates proficiency in statistical inference, covering exact and approximate hypothesis testing for binomial outcomes, normal approximation with continuity correction, multiple-testing adjustments (Bonferroni and Holm), interpretation of p-values, and decision rules for choosing z- versus t-tests, all within the Statistics & Math domain. It is commonly asked to assess understanding of exact versus approximate methods, control of familywise error, and conceptual distinctions about what p-values represent, testing both practical application and conceptual understanding of statistical assumptions and procedures.