PracHub
QuestionsCoachesLearningGuidesInterview Prep
|Home/Machine Learning/CVS Health

Tune classifier and compute key metrics

Last updated: Apr 23, 2026

Quick Overview

This question evaluates practical proficiency in classifier evaluation, threshold tuning, precision/recall trade-offs, cost-sensitive decision rules, and designing hybrid models to predict error amounts within the Machine Learning / Data Science domain.

  • medium
  • CVS Health
  • Machine Learning
  • Data Scientist

Tune classifier and compute key metrics

Company: CVS Health

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

You built a classifier to flag incorrect payments (positive class). On a validation set of 50,000 transactions, the true prevalence of incorrect payments is 2% (1,000). At your current threshold, the model flags 1,800 positives, of which 600 are truly incorrect. A) Compute precision, recall, F1, specificity, false-positive rate, and the confusion-matrix counts (TP, FP, TN, FN). Show arithmetic. B) If the business requires precision ≥ 0.80 while maximizing recall, describe how you would choose the operating threshold using PR curves and cross-validation; specify the selection rule and how you’d report the trade-off. C) In what product scenarios would you prioritize recall over precision, and vice versa, for payment accuracy operations? Give one concrete example each. D) The cost to manually review a flagged transaction is $2; the expected loss from missing an incorrect payment is $200. Propose a cost-sensitive objective or decision rule and explain how it changes thresholding. E) Would you model incorrect-payment detection as classification or regression if you also want to predict the monetary error amount? Outline a hybrid approach (e.g., two-stage model) and justify it.

Quick Answer: This question evaluates practical proficiency in classifier evaluation, threshold tuning, precision/recall trade-offs, cost-sensitive decision rules, and designing hybrid models to predict error amounts within the Machine Learning / Data Science domain.

Related Interview Questions

  • Build a leak-free sklearn churn pipeline - CVS Health (medium)
  • Handle challenges in MMM/MMX - CVS Health (hard)
  • Design classification under missingness and imbalance - CVS Health (hard)
  • Build an uplift model for targeting - CVS Health (hard)
  • Implement R² and Compare PCA With/Without Scaling - CVS Health (medium)
|Home/Machine Learning/CVS Health

Tune classifier and compute key metrics

CVS Health logo
CVS Health
Oct 13, 2025, 9:49 PM
mediumData ScientistTechnical ScreenMachine Learning
6
0
Loading...

Payment Error Classifier — Evaluation, Thresholding, and Cost-Sensitive Design

Context

You built a binary classifier to flag incorrect payments (positive class). On a validation set of 50,000 transactions, the true prevalence of incorrect payments is 2% (1,000). At your current threshold, the model flags 1,800 positives, of which 600 are truly incorrect.

Tasks

A) Compute precision, recall, F1, specificity, false-positive rate, and the confusion-matrix counts (TP, FP, TN, FN). Show arithmetic.

B) If the business requires precision ≥ 0.80 while maximizing recall, describe how you would choose the operating threshold using PR curves and cross-validation; specify the selection rule and how you’d report the trade-off.

C) In what product scenarios would you prioritize recall over precision, and vice versa, for payment accuracy operations? Give one concrete example each.

D) The cost to manually review a flagged transaction is 2;theexpectedlossfrommissinganincorrectpaymentis2; the expected loss from missing an incorrect payment is 2;theexpectedlossfrommissinganincorrectpaymentis200. Propose a cost-sensitive objective or decision rule and explain how it changes thresholding.

E) Would you model incorrect-payment detection as classification or regression if you also want to predict the monetary error amount? Outline a hybrid approach (e.g., two-stage model) and justify it.

Loading comments...

Browse More Questions

More Machine Learning•More CVS Health•More Data Scientist•CVS Health Data Scientist•CVS Health Machine Learning•Data Scientist Machine Learning

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 8,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.