PracHub
QuestionsCoachesLearningGuidesInterview Prep
|Home/Machine Learning/Meta

Evaluate fraud classifier with cost-sensitive metrics

Last updated: Mar 29, 2026

Quick Overview

This question evaluates competency in binary classifier evaluation, cost-sensitive decision-making, probability calibration, and safe online deployment within the Machine Learning domain for a Data Scientist role, requiring practical application informed by conceptual understanding.

  • hard
  • Meta
  • Machine Learning
  • Data Scientist

Evaluate fraud classifier with cost-sensitive metrics

Company: Meta

Role: Data Scientist

Category: Machine Learning

Difficulty: hard

Interview Round: Technical Screen

You inherit a binary fraud classifier. a) Given a holdout confusion matrix TP=200, FP=800, FN=100, TN=99,900 at a specific threshold, compute precision, recall, F1, and false-positive rate. b) If a false negative costs $20 and a false positive costs $0.20 with a 1% prevalence, describe how to choose a threshold using predicted probabilities to maximize expected utility; specify which metric (e.g., PR-AUC vs. ROC-AUC) better reflects improvements at low prevalence and why. c) Outline a calibration check and fix (e.g., reliability plot, isotonic/Platt) and how calibration interacts with threshold selection. d) Propose an online evaluation plan with guardrails to prevent over-blocking legitimate users while improving catch rate (e.g., interleaved shadow evaluation, two-stage review for low-confidence positives), and define success criteria and rollback triggers.

Quick Answer: This question evaluates competency in binary classifier evaluation, cost-sensitive decision-making, probability calibration, and safe online deployment within the Machine Learning domain for a Data Scientist role, requiring practical application informed by conceptual understanding.

Related Interview Questions

  • Self-Attention: Implementation, Complexity, and Efficient Variants - Meta (hard)
  • Machine Learning Fundamentals: Optimizers, Scaling Laws, and Clustering - Meta (hard)
  • Implement 1NN Embeddings and Forward Pass - Meta (hard)
  • Design and evaluate an ads ranking algorithm - Meta (easy)
  • How would you design a Shop Ads ranking algorithm? - Meta (easy)
|Home/Machine Learning/Meta

Evaluate fraud classifier with cost-sensitive metrics

Meta logo
Meta
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenMachine Learning
2
0

Binary Fraud Classifier: Metrics, Thresholding, Calibration, and Online Evaluation

You inherit a binary fraud classifier used to decide whether to block an event. At one operating threshold, the holdout confusion matrix is:

  • TP = 200, FP = 800, FN = 100, TN = 99,900

Answer the following:

(a) Compute precision, recall, F1, and false-positive rate (FPR) at this threshold.

(b) Assuming a false negative costs 20andafalsepositivecosts20 and a false positive costs 20andafalsepositivecosts0.20 in an environment with 1% fraud prevalence, describe how to choose a threshold using predicted probabilities to maximize expected utility. State which ranking metric (PR-AUC vs. ROC-AUC) better reflects improvements at low prevalence and why.

(c) Outline how you would check and fix probability calibration (e.g., reliability plot, isotonic or Platt scaling) and explain how calibration interacts with threshold selection.

(d) Propose an online evaluation plan with guardrails that avoids over-blocking legitimate users while improving catch rate. Include ideas such as shadow evaluation and a two-stage review for low-confidence positives, and define concrete success criteria and rollback triggers.

Loading comments...

Browse More Questions

More Machine Learning•More Meta•More Data Scientist•Meta Data Scientist•Meta Machine Learning•Data Scientist Machine Learning

Write your answer

Your first approved answer each day earns 20 XP.

Sign in to write your answer.
PracHub

Master your tech interviews with 8,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • AI Coding Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.