Explain ROC-AUC vs PR-AUC tradeoffs
Company: Databricks
Role: Data Scientist
Category: Machine Learning
Difficulty: hard
Interview Round: Technical Screen
In a binary classification setting, you trained a model that outputs predicted probabilities.
1) Explain what ROC-AUC and PR-AUC measure, including the axes/curves and the intuition behind each.
2) When can ROC-AUC look “good” while PR-AUC looks “poor” (or vice versa)? Discuss the role of class imbalance and base rate.
3) If your business goal is to catch as many positives as possible while reviewing a limited number of cases (e.g., fraud investigation), which metric/curve would you emphasize and why?
4) Name at least two practical pitfalls/edge cases when using these metrics (e.g., calibration vs ranking, operating-threshold selection, changing prevalence between train and production).
Quick Answer: This question evaluates understanding of binary classification evaluation metrics—specifically interpretation and trade-offs between ROC-AUC and PR-AUC—testing conceptual skills in model ranking, sensitivity to class imbalance, and selection of appropriate performance measures within the Machine Learning domain.