Choose Classification Metrics Under Asymmetric Costs

Quick Overview

This question evaluates understanding of binary classification evaluation metrics and model-operating decisions, covering confusion-matrix-derived measures (precision, recall/sensitivity, specificity, false positive/negative rates), summary metrics (ROC-AUC, PR-AUC), Type I/II errors, threshold selection, prevalence drift, probability calibration, and cost-sensitive trade-offs. It is commonly asked to assess a data scientist's ability to reason about model performance under class imbalance and asymmetric business costs, and it belongs to the Statistics & Math / Machine Learning model-evaluation domain requiring both conceptual understanding and practical application.

Choose Classification Metrics Under Asymmetric Costs

Company: Microsoft

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Technical Screen

You are evaluating a binary classification model for a business problem. Explain how to use a confusion matrix to compute and interpret: - precision, - recall (sensitivity), - specificity, - false positive rate, - false negative rate, - ROC-AUC, - PR-AUC. Also answer the following: 1. What is the difference between Type I and Type II error? 2. How does your metric choice change across scenarios such as spam filtering, medical screening, fraud detection, and ad click prediction? 3. Why can accuracy be misleading under class imbalance? 4. How do decision thresholds, prevalence drift, and probability calibration affect model evaluation? 5. If false positives and false negatives have different business costs, how would you choose an operating threshold?

Quick Answer: This question evaluates understanding of binary classification evaluation metrics and model-operating decisions, covering confusion-matrix-derived measures (precision, recall/sensitivity, specificity, false positive/negative rates), summary metrics (ROC-AUC, PR-AUC), Type I/II errors, threshold selection, prevalence drift, probability calibration, and cost-sensitive trade-offs. It is commonly asked to assess a data scientist's ability to reason about model performance under class imbalance and asymmetric business costs, and it belongs to the Statistics & Math / Machine Learning model-evaluation domain requiring both conceptual understanding and practical application.

|Home/Statistics & Math/Microsoft
Microsoft logo
Microsoft
Feb 25, 2026, 12:00 AM
mediumData ScientistTechnical ScreenStatistics & Math
11
0

You are evaluating a binary classification model for a business problem.

Explain how to use a confusion matrix to compute and interpret:

  • precision,
  • recall (sensitivity),
  • specificity,
  • false positive rate,
  • false negative rate,
  • ROC-AUC,
  • PR-AUC.

Also answer the following:

  1. What is the difference between Type I and Type II error?
  2. How does your metric choice change across scenarios such as spam filtering, medical screening, fraud detection, and ad click prediction?
  3. Why can accuracy be misleading under class imbalance?
  4. How do decision thresholds, prevalence drift, and probability calibration affect model evaluation?
  5. If false positives and false negatives have different business costs, how would you choose an operating threshold?
Loading comments...

Browse More Questions

More Statistics & Math•More Microsoft•More Data Scientist•Microsoft Data Scientist•Microsoft Statistics & Math•Data Scientist Statistics & Math