PracHub
QuestionsPremiumLearningGuidesInterview PrepNEWCoaches
|Home/Machine Learning/OneMain Financial

Choose evaluation metrics for imbalanced risk model

Last updated: Mar 29, 2026

Quick Overview

This question evaluates understanding of cost-sensitive decision making, probabilistic thresholding, evaluation metric selection under class imbalance, constrained operating points for manual review caps, and probability calibration.

  • medium
  • OneMain Financial
  • Machine Learning
  • Data Scientist

Choose evaluation metrics for imbalanced risk model

Company: OneMain Financial

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

You build a fraud detector with 1% positives. Business costs: false negative = $100, false positive = $1; true positives/true negatives have zero cost. 1) Derive the Bayes-optimal probability threshold for a calibrated classifier that minimizes expected cost and compute its numeric value. 2) Decide whether ROC-AUC, PR-AUC, F1, MCC, KS, or cost-based metrics best reflect business goals and justify. 3) Describe how you would choose a threshold on a validation set to maximize expected profit while ensuring a cap of ≤0.5% manual review rate. 4) Explain how you would verify calibration (e.g., reliability diagrams, Brier score) and recalibrate (Platt vs isotonic) without leakage.

Quick Answer: This question evaluates understanding of cost-sensitive decision making, probabilistic thresholding, evaluation metric selection under class imbalance, constrained operating points for manual review caps, and probability calibration.

Related Interview Questions

  • Explain decision trees and tree ensembles - OneMain Financial (easy)
  • Handle missing data and outliers robustly - OneMain Financial (hard)
  • Select and tune XGBoost hyperparameters - OneMain Financial (hard)
  • Handle Missing Values and Outliers in Machine Learning - OneMain Financial (medium)
OneMain Financial logo
OneMain Financial
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Machine Learning
3
0
Loading...

Cost-Sensitive Fraud Detection: Thresholding, Metrics, and Calibration

Assume a binary fraud classifier outputs calibrated probabilities p = P(y=1|x). The base rate of fraud is 1%. Business costs:

  • False negative (missed fraud): $100
  • False positive (flagging a non-fraud): $1
  • True positives and true negatives: $0

Answer the following:

  1. Derive the Bayes-optimal probability threshold that minimizes expected cost for a calibrated classifier, and compute its numeric value with the given costs.
  2. Among ROC-AUC, PR-AUC, F1, MCC, KS, or cost-based metrics, identify which best reflect these business goals and justify your choice.
  3. You must cap manual reviews to ≤0.5% of all cases. Describe how to choose a validation-set threshold that maximizes expected profit (minimizes expected cost) subject to that cap.
  4. Explain how to verify calibration (e.g., reliability diagrams, Brier score) and how to recalibrate (Platt scaling vs. isotonic regression) without data leakage.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Machine Learning•More OneMain Financial•More Data Scientist•OneMain Financial Data Scientist•OneMain Financial Machine Learning•Data Scientist Machine Learning
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.