PracHub
QuestionsPremiumLearningGuidesInterview PrepNEWCoaches
|Home/Machine Learning/CVS Health

Tune classifier and compute key metrics

Last updated: Apr 23, 2026

Quick Overview

This question evaluates practical proficiency in classifier evaluation, threshold tuning, precision/recall trade-offs, cost-sensitive decision rules, and designing hybrid models to predict error amounts within the Machine Learning / Data Science domain.

  • medium
  • CVS Health
  • Machine Learning
  • Data Scientist

Tune classifier and compute key metrics

Company: CVS Health

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

You built a classifier to flag incorrect payments (positive class). On a validation set of 50,000 transactions, the true prevalence of incorrect payments is 2% (1,000). At your current threshold, the model flags 1,800 positives, of which 600 are truly incorrect. A) Compute precision, recall, F1, specificity, false-positive rate, and the confusion-matrix counts (TP, FP, TN, FN). Show arithmetic. B) If the business requires precision ≥ 0.80 while maximizing recall, describe how you would choose the operating threshold using PR curves and cross-validation; specify the selection rule and how you’d report the trade-off. C) In what product scenarios would you prioritize recall over precision, and vice versa, for payment accuracy operations? Give one concrete example each. D) The cost to manually review a flagged transaction is $2; the expected loss from missing an incorrect payment is $200. Propose a cost-sensitive objective or decision rule and explain how it changes thresholding. E) Would you model incorrect-payment detection as classification or regression if you also want to predict the monetary error amount? Outline a hybrid approach (e.g., two-stage model) and justify it.

Quick Answer: This question evaluates practical proficiency in classifier evaluation, threshold tuning, precision/recall trade-offs, cost-sensitive decision rules, and designing hybrid models to predict error amounts within the Machine Learning / Data Science domain.

Related Interview Questions

  • Build a leak-free sklearn churn pipeline - CVS Health (medium)
  • Handle challenges in MMM/MMX - CVS Health (hard)
  • Design classification under missingness and imbalance - CVS Health (hard)
  • Build an uplift model for targeting - CVS Health (hard)
  • Implement R² and Compare PCA With/Without Scaling - CVS Health (medium)
CVS Health logo
CVS Health
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Machine Learning
5
0
Loading...

Payment Error Classifier — Evaluation, Thresholding, and Cost-Sensitive Design

Context

You built a binary classifier to flag incorrect payments (positive class). On a validation set of 50,000 transactions, the true prevalence of incorrect payments is 2% (1,000). At your current threshold, the model flags 1,800 positives, of which 600 are truly incorrect.

Tasks

A) Compute precision, recall, F1, specificity, false-positive rate, and the confusion-matrix counts (TP, FP, TN, FN). Show arithmetic.

B) If the business requires precision ≥ 0.80 while maximizing recall, describe how you would choose the operating threshold using PR curves and cross-validation; specify the selection rule and how you’d report the trade-off.

C) In what product scenarios would you prioritize recall over precision, and vice versa, for payment accuracy operations? Give one concrete example each.

D) The cost to manually review a flagged transaction is 2;theexpectedlossfrommissinganincorrectpaymentis2; the expected loss from missing an incorrect payment is 2;theexpectedlossfrommissinganincorrectpaymentis200. Propose a cost-sensitive objective or decision rule and explain how it changes thresholding.

E) Would you model incorrect-payment detection as classification or regression if you also want to predict the monetary error amount? Outline a hybrid approach (e.g., two-stage model) and justify it.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Machine Learning•More CVS Health•More Data Scientist•CVS Health Data Scientist•CVS Health Machine Learning•Data Scientist Machine Learning
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.