A payment-accuracy team launched a rule intended to reduce incorrect payments. In a prior period (control), 10,000 transactions yielded 9,520 correct and 480 incorrect. In the post-launch period (treatment), 10,000 transactions yielded 9,640 correct and 360 incorrect. 1) Choose an appropriate hypothesis test and state H0 and H1 formally. 2) Compute the test statistic and two-sided p-value for the difference in accuracy (proportions). Show the pooled vs unpooled choices and justify which you use. 3) Construct a 95% confidence interval for the difference in proportions (treatment − control). 4) Interpret the results for a non-technical stakeholder in one sentence, including practical significance. 5) Power planning: If baseline accuracy is 96.0%, what equal per-group sample size is needed to detect an absolute improvement of 0.2 percentage points (to 96.2%) with 80% power at α = 0.05 (two-sided), using the normal approximation? Show the formula you use and the numeric inputs (Z-scores).

This question evaluates proficiency in hypothesis testing for proportions, interpretation of p-values and confidence intervals, and power/sample-size calculations within the Statistics & Math domain for a Data Scientist role.

How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

What difficulty level is this interview question?

This is a medium difficulty Statistics & Math question, commonly asked during Technical Screen rounds at CVS Health.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at CVS Health during technical interviews.

Test payment-accuracy lift with p-value and power

AB Test of Payment Accuracy (Two Proportions)

Assume transactions in the two periods are independent and large enough for normal approximations. Define "success" as a correct payment (accuracy).

Control (pre-launch): n = 10,000, correct = 9,520, incorrect = 480
Treatment (post-launch): n = 10,000, correct = 9,640, incorrect = 360

Tasks

Choose an appropriate hypothesis test and state H0 and H1 formally.
Compute the test statistic and two-sided p-value for the difference in accuracy (proportions). Show pooled vs unpooled standard error choices and justify which you use for the test.
Construct a 95% confidence interval (CI) for the difference in proportions (treatment − control).
Interpret the results for a non-technical stakeholder in one sentence, including practical significance.
Power planning: If baseline accuracy is 96.0%, what equal per-group sample size is needed to detect an absolute improvement of 0.2 percentage points (to 96.2%) with 80% power at α = 0.05 (two-sided), using the normal approximation? Show the formula and numeric inputs (Z-scores).

AB Test of Payment Accuracy (Two Proportions)

Assume transactions in the two periods are independent and large enough for normal approximations. Define "success" as a correct payment (accuracy).

Control (pre-launch): n = 10,000, correct = 9,520, incorrect = 480

Treatment (post-launch): n = 10,000, correct = 9,640, incorrect = 360

Tasks

Choose an appropriate hypothesis test and state H0 and H1 formally.

Compute the test statistic and two-sided p-value for the difference in accuracy (proportions). Show pooled vs unpooled standard error choices and justify which you use for the test.

Construct a 95% confidence interval (CI) for the difference in proportions (treatment − control).

Interpret the results for a non-technical stakeholder in one sentence, including practical significance.

Power planning: If baseline accuracy is 96.0%, what equal per-group sample size is needed to detect an absolute improvement of 0.2 percentage points (to 96.2%) with 80% power at α = 0.05 (two-sided), using the normal approximation? Show the formula and numeric inputs (Z-scores).

Test payment-accuracy lift with p-value and power

Quick Overview

AB Test of Payment Accuracy (Two Proportions)

Solution

Comments (0)

Test payment-accuracy lift with p-value and power

Quick Overview

AB Test of Payment Accuracy (Two Proportions)

Solution

Comments (0)