Explain and interpret p-values correctly
Overview
This question evaluates statistical inference and experimental-design competencies, focusing on a precise definition and correct interpretation of p-values along with how sample size, effect size, variance, multiple testing, sequential peeking, and imbalanced or delayed labels affect experimental conclusions in a fraud-detection context.
PayPal
Jan 17, 2026, 12:00 AM
Data Scientist
Technical Screen
Statistics & Math
0
0
Loading...
Context
You are evaluating a change to a fraud decision rule (e.g., a new threshold or step-up authentication rule). You run an experiment comparing Control vs Treatment.
Questions
- What is a p-value ? State the definition precisely.
- What does a p-value not tell you (common misinterpretations)?
- How do sample size , effect size , and variance influence p-values?
-
How would you handle:
- Multiple testing (many metrics or many segments like regions)
- Peeking (checking results daily and stopping early)
- Imbalanced outcomes and delayed labels (e.g., chargebacks arriving weeks later)
- What would you report alongside the p-value to make a decision in a fraud setting (confidence intervals, practical significance, cost-based impact, etc.)?