PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Statistics & Math/Amazon

Identify P-Value Limitations and Complementary Approaches

Last updated: Mar 29, 2026

Quick Overview

This question evaluates understanding of statistical inference and experimental analysis, focusing on the limitations of p-values and the identification of complementary metrics and practices in A/B testing within the Statistics & Math domain.

  • medium
  • Amazon
  • Statistics & Math
  • Data Scientist

Identify P-Value Limitations and Complementary Approaches

Company: Amazon

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Onsite

##### Scenario During an A/B test for a new product feature, stakeholders rely on p-values to decide shipment. ##### Question Explain at least three key limitations of p-values in hypothesis testing and suggest complementary measures or approaches that mitigate these issues. ##### Hints Think about effect size, sample size dependence, multiple testing, practical significance, prior beliefs, and p-hacking.

Quick Answer: This question evaluates understanding of statistical inference and experimental analysis, focusing on the limitations of p-values and the identification of complementary metrics and practices in A/B testing within the Statistics & Math domain.

Related Interview Questions

  • Compute an A/B test p-value by hand - Amazon (medium)
  • Compute and interpret quantile loss vs RMSE - Amazon (medium)
  • Compute CIs, power, and multiple testing - Amazon (medium)
  • Plan and analyze an A/B test - Amazon (hard)
  • Compute p-values, CIs, and adjust multiples - Amazon (Medium)
Amazon logo
Amazon
Aug 4, 2025, 10:55 AM
Data Scientist
Onsite
Statistics & Math
16
0

A/B Testing: Limits of P-values and Better Decision Practices

Scenario

Your team is running an A/B test for a new product feature and stakeholders rely on p-values to decide whether to ship.

Task

Explain at least three key limitations of p-values in hypothesis testing and suggest complementary measures or approaches that mitigate these issues. Use A/B testing context and brief examples where helpful.

Guidance

Consider topics such as: effect size, dependence on sample size, multiple testing, practical significance, prior beliefs, and p-hacking/data peeking.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Statistics & Math•More Amazon•More Data Scientist•Amazon Data Scientist•Amazon Statistics & Math•Data Scientist Statistics & Math
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.