Identify and Fix Predictive Model Performance Gaps
Quick Overview
Evaluates predictive-model audit skills across month encoding, feature scaling, and imbalanced classification. Strong answers fix cyclical seasonality, standardize where needed, and adjust losses or thresholds for recall.
Identify and Fix Predictive Model Performance Gaps
Company: Google
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
Reviewing an existing predictive model for operational issues and performance gaps.
##### Question
A model treats calendar month as a continuous variable. What problems can this cause and how would you fix them? Why is standardizing predictors important before fitting certain models, and what might go wrong if you skip it? Your training data are highly imbalanced. Describe two ways to adjust the loss function or evaluation metrics so recall is properly rewarded.
##### Hints
Discuss cyclic features, scale sensitivity, weighted loss, focal loss, precision-recall trade-off.
Quick Answer: Evaluates predictive-model audit skills across month encoding, feature scaling, and imbalanced classification. Strong answers fix cyclical seasonality, standardize where needed, and adjust losses or thresholds for recall.
Model Review: Month Encoding, Feature Scaling, and Imbalanced Data
You are auditing an existing predictive model for operational performance. The current implementation encodes calendar month as a continuous variable and is trained on imbalanced data.
Address the modeling issues below and propose concrete fixes.
Constraints & Assumptions
The model may be linear, distance-based, tree-based, or another supervised model; specify where your advice depends on model class.
Treat month as a cyclical calendar feature, not a true continuous magnitude.
Connect preprocessing choices to validation performance and production behavior.
For imbalance, discuss both training-time and threshold or evaluation-time changes.
Clarifying Questions to Ask Guidance
What model class is currently used?
What is the target variable, class prevalence, and cost of false positives versus false negatives?
Are there seasonal effects beyond month, such as holidays or day of week?
Which metric matters most: recall, precision, F1, PR-AUC, cost, or calibration?
Part 1 - Month as a Continuous Variable
What problems arise if the model treats month as Jan = 1 through Dec = 12? How would you fix it?
What This Part Should Cover Guidance
Explain cyclicity: December and January are adjacent, but numeric encoding makes them far apart.
Explain monotonicity and arbitrary-distance problems for linear and distance-based models.
Propose one-hot encoding, cyclic sine/cosine encoding, Fourier features, target encoding with care, or holiday features.
Validate the fix with time-aware splits or backtests.
Part 2 - Feature Standardization
Why is standardizing predictors important before fitting certain models, and what can go wrong if it is skipped?
What This Part Should Cover Guidance
Explain that scale affects regularized linear models, kNN, SVMs, PCA, gradient-based optimization, and neural networks.
Note that tree-based models usually do not require standardization for split quality.
Describe risks such as coefficient comparability issues, slow convergence, domination of large-scale features, and unstable regularization.
Fit scalers on training data only to avoid leakage.
Part 3 - Imbalanced Data and Recall
Describe concrete ways to adjust training, evaluation, or thresholds so recall is properly rewarded.
What This Part Should Cover Guidance
Include class weights, focal loss, weighted loss, oversampling, undersampling, or synthetic sampling where appropriate.
Tune decision thresholds using PR curves, recall targets, cost curves, or business constraints.
Use metrics such as recall, precision at recall, PR-AUC, F-beta, confusion matrix, and calibration by segment.
Explain trade-offs with false positives and operational capacity.
Follow-up Questions Guidance
When would one-hot month encoding be better than sine/cosine encoding?
How would you avoid leakage when standardizing features in cross-validation?
What would you do if improving recall doubles the false-positive workload?