How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a hard difficulty Machine Learning question, commonly asked during Technical Screen rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Choose ML metrics under asymmetric costs

Quick Overview

This question evaluates a data scientist's competency in cost-sensitive binary classification, covering skills such as defining business cost matrices, threshold selection and probability calibration, handling extreme class imbalance, fairness assessment across segments, monitoring for drift and feedback loops, and designing experiments to measure long-term product impact. It is a machine learning domain question commonly asked to probe practical judgment about trade-offs between precision and recall, operational metrics, and product-level consequences, testing both conceptual understanding of decision theory and practical application to production ML.

Binary Classifier With Asymmetric Costs: Fraud vs. Cancer

Context: You own a production binary classifier and must make product/ML decisions under asymmetric error costs. Compare two use cases:

(A) Credit-card fraud detection
(B) Cancer detection (screening/triage)

Tasks:

For each use case, define a business cost matrix (TP, FP, FN, TN) and state the metric(s) you would optimize (e.g., expected cost, recall at fixed precision, PR AUC). Explain why ROC AUC may be misleading in these settings.
Describe how you would set thresholds using cost curves or iso-F1/iso-precision lines. Explain how probability calibration (Platt scaling or isotonic regression) changes the decision rule and threshold.
Explain how you would handle extreme class imbalance (e.g., sampling, class weights, focal loss) and evaluate with stratified, time-aware cross-validation.
Discuss fairness and the distribution of false-positive burden by segment; propose how to assess and mitigate disparities.
List online metrics and logs to monitor drift and feedback loops after launch.
Explain how short-term metric gains might reduce long-term product engagement, and propose an experiment to measure long-term impact.

Quick Overview

Binary Classifier With Asymmetric Costs: Fraud vs. Cancer

Context: You own a production binary classifier and must make product/ML decisions under asymmetric error costs. Compare two use cases:

(A) Credit-card fraud detection

(B) Cancer detection (screening/triage)

Tasks:

For each use case, define a business cost matrix (TP, FP, FN, TN) and state the metric(s) you would optimize (e.g., expected cost, recall at fixed precision, PR AUC). Explain why ROC AUC may be misleading in these settings.

Describe how you would set thresholds using cost curves or iso-F1/iso-precision lines. Explain how probability calibration (Platt scaling or isotonic regression) changes the decision rule and threshold.

Explain how you would handle extreme class imbalance (e.g., sampling, class weights, focal loss) and evaluate with stratified, time-aware cross-validation.

Discuss fairness and the distribution of false-positive burden by segment; propose how to assess and mitigate disparities.

List online metrics and logs to monitor drift and feedback loops after launch.

Explain how short-term metric gains might reduce long-term product engagement, and propose an experiment to measure long-term impact.

Choose ML metrics under asymmetric costs

Quick Overview

Binary Classifier With Asymmetric Costs: Fraud vs. Cancer

Solution

Submit Your Answer to Earn 20XP

Choose ML metrics under asymmetric costs

Quick Overview

Binary Classifier With Asymmetric Costs: Fraud vs. Cancer

Solution

Submit Your Answer to Earn 20XP