Explain MSE vs MAE, AUC, and imbalance handling
Company: Capital One
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: HR Screen
Answer all parts concisely and precisely. 1) When would you prefer MAE over MSE in a regression objective? Compare robustness to outliers, gradient behavior near zero, and optimization consequences; give a concrete example where MSE underperforms while MAE is acceptable. 2) For a binary classifier with 1% positive prevalence, interpret ROC‑AUC = 0.90 and PR‑AUC = 0.25. Which metric is more informative here and why? Explain how ROC can look strong while PR remains weak; reference score distributions that cause this. 3) For a neural network binary classifier, choose an output activation and loss. Explain how you would handle class imbalance using class weights or focal loss, and describe how each changes gradient contributions of positive vs. negative examples. 4) If the business requires precision ≥ 0.50, describe exactly how you would pick a probability threshold on validation data and avoid optimistic bias (e.g., nested CV or a hold‑out).
Quick Answer: This question evaluates a candidate's understanding of regression loss trade-offs (MAE vs MSE), classification evaluation metrics under severe class imbalance (ROC-AUC vs PR-AUC), neural network output activation and loss dynamics, and practical techniques for imbalance mitigation and probability-threshold selection in binary classification.
