Explain p-value and choose correct test

Quick Overview

This question evaluates a candidate's understanding of statistical inference, hypothesis testing, and robust test selection, focusing on p-value interpretation and choosing between paired vs independent and parametric vs rank-based tests.

Explain p-value and choose correct test

Company: CVS Health

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Technical Screen

Part A — Plain-language p-value: Explain a p-value to a non-scientist without using the phrase 'probability the null is true.' Use a concrete everyday analogy and clarify common misinterpretations (e.g., p≠effect size, p is conditional on the test and assumptions). Part B — Wilcoxon vs t-test: For each scenario, select and justify the appropriate test, assumptions, and an effect size measure. 1) Paired data: 12 patients' systolic BP measured pre/post a low-sodium diet; the distribution of paired differences is skewed with outliers. Choose between a paired t-test and a Wilcoxon signed-rank test. State assumptions, how you'd check them, how ties/zeros are handled, and report an effect size (e.g., Cohen's dz vs rank-biserial or matched-pairs r). 2) Independent samples: Compare length of stay between two independent clinics (n=18 and n=25) with unequal variances and non-normal heavy tails. Choose between Welch's t-test and a Wilcoxon rank-sum (Mann–Whitney) test. Discuss what the nonparametric test estimates (probability of superiority) vs the mean-difference focus of Welch's test, when each is preferable, and how you'd complement with confidence intervals and a robust effect size (e.g., Hedges' g with HC3 SEs or Cliff's delta).

Quick Answer: This question evaluates a candidate's understanding of statistical inference, hypothesis testing, and robust test selection, focusing on p-value interpretation and choosing between paired vs independent and parametric vs rank-based tests.

|Home/Statistics & Math/CVS Health
CVS Health logo
CVS Health
Oct 13, 2025
mediumData ScientistTechnical ScreenStatistics & Math
5
0

Technical Screen: P-values and Robust Two-Sample/Paired Tests

Context: You are a data scientist evaluating healthcare interventions. Answer in clear, interview-ready explanations that a stakeholder could understand and that a peer could reproduce.

Part A — Plain-language p-value

Explain a p-value to a non-scientist without using the phrase "probability the null is true." Use a concrete everyday analogy and clarify common misinterpretations, including:

  • p is not an effect size.
  • p depends on the chosen test, analysis plan, and assumptions.
  • A non-significant p does not prove no effect, and a significant p does not prove practical importance.

Part B — Wilcoxon vs t-test

For each scenario, select and justify the appropriate test, the assumptions, how you would check them, and an effect size measure. Be explicit about handling ties/zeros for rank tests and how you would report confidence intervals.

  1. Paired data (n = 12): Systolic BP measured pre/post a low-sodium diet. The distribution of paired differences is skewed with outliers. Choose between a paired t-test and a Wilcoxon signed-rank test. State:
    • Assumptions and how you'd check them.
    • How ties/zero differences are handled.
    • An appropriate effect size (e.g., Cohen's dz vs rank-biserial or matched-pairs r) and CI.
  2. Independent samples (n1 = 18, n2 = 25): Compare length of stay between two clinics with unequal variances and heavy-tailed, non-normal distributions. Choose between Welch's t-test and Wilcoxon rank-sum (Mann–Whitney). Discuss:
    • What Mann–Whitney estimates (probability of superiority) vs Welch's mean-difference focus.
    • When each is preferable given goals and data features.
    • How you'd complement with confidence intervals and a robust effect size (e.g., Hedges' g with HC3 SEs or Cliff's delta).
Loading comments...