Diagnose and fix flawed model fit
Company: Google
Role: Data Scientist
Category: Machine Learning
Difficulty: hard
Interview Round: Technical Screen
You are handed an existing binary classifier (predicting churn=1) with the following issues: (a) the feature “month” is encoded as an integer 1–12 and used linearly; (b) continuous features of wildly different scales were not standardized; (c) positives are 3% of the data; (d) model is evaluated by overall accuracy. The business requires Recall ≥ 80% at Precision ≥ 30%.
- Identify and explain the modeling mistakes. For month, propose and justify two encodings (e.g., one-hot vs. cyclical sin/cos) and when you’d prefer each.
- Describe a training procedure to address scale and imbalance: feature standardization, class-weighted loss vs. resampling (SMOTE vs. undersampling), and how you’d select the weight ratio or sampling rate without biasing validation.
- Specify an evaluation protocol: stratified cross-validation, primary metrics (PR AUC, recall@precision≥0.30), calibration assessment, and threshold selection to satisfy the operating point. Include how you’d compute and report confidence intervals for these metrics.
- If the dataset is imbalanced across subgroups, define a cost-sensitive or group-weighted objective and the fairness/coverage checks you’d add before deployment. Provide a concrete example of a failure mode you would catch and how you’d mitigate it.
Quick Answer: This question evaluates a data scientist's competency in applied supervised learning diagnostics, including feature encoding, feature scaling, class imbalance mitigation, evaluation metric selection, calibration, and fairness-aware objective design.