You own a credit-card fraud classifier deployed as a probability scorer. Choose an operating threshold under asymmetric costs and justify it quantitatively. Assume per 1,000,000 transactions: base fraud rate = 0.20%. Costs: False Positive (decline a good transaction) = $15; False Negative (missed fraud) = $100. Consider three candidate thresholds with the following operating points on a representative validation set: - T1: TPR = 0.90, FPR = 0.020 - T2: TPR = 0.80, FPR = 0.010 - T3: TPR = 0.65, FPR = 0.004 Tasks: - For each threshold, compute the expected number of TP, FP, FN, TN and the expected total cost. Pick the threshold that minimizes expected cost and explain the business trade-offs. - Explain how you would calibrate scores (e.g., Platt/Isotonic), monitor for dataset/label drift, and periodically re-tune the threshold by segment (country, merchant category, transaction amount). - Propose guardrail metrics to detect harmful side effects (e.g., surge in declines for high-LTV users) and an experiment to validate changes without causing unacceptable customer pain.

This question evaluates cost-sensitive classification, operating-threshold selection, score calibration, drift monitoring, and experiment and guardrail design competencies within the Machine Learning domain.

How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a medium difficulty Machine Learning question, commonly asked during Onsite rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Choose threshold under asymmetric costs

You own a credit-card fraud classifier deployed as a probability scorer. Choose an operating threshold under asymmetric costs and justify it quantitatively. Assume per 1,000,000 transactions: base fraud rate = 0.20%. Costs: False Positive (decline a good transaction) = $15; False Negative (missed fraud) =$ 100. Consider three candidate thresholds with the following operating points on a representative validation set:

T1: TPR = 0.90, FPR = 0.020
T2: TPR = 0.80, FPR = 0.010
T3: TPR = 0.65, FPR = 0.004 Tasks:
For each threshold, compute the expected number of TP, FP, FN, TN and the expected total cost. Pick the threshold that minimizes expected cost and explain the business trade-offs.
Explain how you would calibrate scores (e.g., Platt/Isotonic), monitor for dataset/label drift, and periodically re-tune the threshold by segment (country, merchant category, transaction amount).
Propose guardrail metrics to detect harmful side effects (e.g., surge in declines for high-LTV users) and an experiment to validate changes without causing unacceptable customer pain.

T1: TPR = 0.90, FPR = 0.020
T2: TPR = 0.80, FPR = 0.010
T3: TPR = 0.65, FPR = 0.004 Tasks:
For each threshold, compute the expected number of TP, FP, FN, TN and the expected total cost. Pick the threshold that minimizes expected cost and explain the business trade-offs.
Explain how you would calibrate scores (e.g., Platt/Isotonic), monitor for dataset/label drift, and periodically re-tune the threshold by segment (country, merchant category, transaction amount).
Propose guardrail metrics to detect harmful side effects (e.g., surge in declines for high-LTV users) and an experiment to validate changes without causing unacceptable customer pain.

Choose threshold under asymmetric costs

Quick Overview

Choose threshold under asymmetric costs

Write your answer

Choose threshold under asymmetric costs

Quick Overview

Choose threshold under asymmetric costs

Write your answer