Resolve a Mismatch Between Regression and Classification Metrics
Quick Overview
Resolve an interview prompt that mixes regression with precision and recall. Build separate plans for rare-event classification and skewed continuous targets, with metrics tied to the real decision.
Resolve a Mismatch Between Regression and Classification Metrics
Company: Capital One
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Online Assessment
An assessment asks you to “train a regressor on imbalanced data and evaluate it with precision and recall.” Explain why that request is internally ambiguous and how you would resolve it before modeling. Give a complete plan for each plausible interpretation: a rare-event classification target and a genuinely continuous regression target.
### Constraints & Assumptions
- Do not relabel a continuous outcome solely to make precision and recall available without discussing information loss.
- Any resampling must occur only within training data, never before the validation split.
- The evaluation must match the decision the model will support.
### Clarifying Questions to Ask
- Is the target categorical, binary, ordinal, or continuous?
- What real-world action follows a positive prediction or a numeric estimate?
- What are the costs of false positives, false negatives, and large numeric errors?
- Does “imbalanced” refer to rare classes or a skewed continuous target distribution?
```hint Start with the estimand
Model family and metrics follow from the target and decision; the words “regressor” and “precision” cannot both define the task without clarification.
```
### What a Strong Answer Covers
- Identification of the regression/classification mismatch.
- A rare-event classification workflow with suitable splits, thresholds, and metrics.
- A continuous-target workflow with regression losses and tail-aware diagnostics.
- Leakage controls for weighting, resampling, tuning, and model selection.
- A recommendation tied to operational costs rather than a single default score.
### Follow-up Questions
- When is average precision more informative than ROC AUC?
- How would probability calibration change the threshold decision?
- How can a regression model be evaluated specifically on rare high-value outcomes without binarizing the target?
Quick Answer: Resolve an interview prompt that mixes regression with precision and recall. Build separate plans for rare-event classification and skewed continuous targets, with metrics tied to the real decision.
An assessment asks you to “train a regressor on imbalanced data and evaluate it with precision and recall.” Explain why that request is internally ambiguous and how you would resolve it before modeling. Give a complete plan for each plausible interpretation: a rare-event classification target and a genuinely continuous regression target.
Constraints & Assumptions
Do not relabel a continuous outcome solely to make precision and recall available without discussing information loss.
Any resampling must occur only within training data, never before the validation split.
The evaluation must match the decision the model will support.
Clarifying Questions to Ask Guidance
Is the target categorical, binary, ordinal, or continuous?
What real-world action follows a positive prediction or a numeric estimate?
What are the costs of false positives, false negatives, and large numeric errors?
Does “imbalanced” refer to rare classes or a skewed continuous target distribution?
What a Strong Answer Covers Guidance
Identification of the regression/classification mismatch.
A rare-event classification workflow with suitable splits, thresholds, and metrics.
A continuous-target workflow with regression losses and tail-aware diagnostics.
Leakage controls for weighting, resampling, tuning, and model selection.
A recommendation tied to operational costs rather than a single default score.
Follow-up Questions Guidance
When is average precision more informative than ROC AUC?
How would probability calibration change the threshold decision?
How can a regression model be evaluated specifically on rare high-value outcomes without binarizing the target?