Handle imbalance, validate samples, and avoid overfitting
Quick Overview
This question evaluates competencies in handling class imbalance, choosing and interpreting evaluation metrics and decision thresholds, validating sample representativeness and model generalization from very large datasets, mitigating overfitting in decision-tree and ensemble models, and understanding how L1/L2 regularization introduces bias, all within the Machine Learning domain for Data Scientist roles. It is commonly asked to assess both practical application skills—such as model validation, sampling and hyperparameter controls—and conceptual understanding of bias–variance and regularization trade-offs, indicating readiness for production-grade supervised learning problems.
Answer the following applied ML questions.
1) Class imbalance
You’re building a binary classifier where positives are rare.
- What are practical ways to handle class imbalance?
- Which evaluation metrics would you use and why (e.g., precision/recall/F1, ROC-AUC, PR-AUC)?
- How would you pick a decision threshold?
2) Training on a sample from a huge dataset
You have an extremely large dataset, so you train on a sample.
- How do you verify the sampled dataset is representative of the full dataset?
- How do you verify that a model trained on the sample will generalize to the full distribution?
3) Prevent overfitting in tree-based models
For decision trees / gradient-boosted trees / random forests:
- What are the main knobs/strategies to reduce overfitting?
- What validation approach would you use?
4) Why aren’t L1/L2 regularized estimators unbiased?
Explain why adding L1 (lasso) or L2 (ridge) regularization introduces bias, and why it can still improve generalization.