Explain overfitting and how to prevent it
Company: Pinterest
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: hard
Interview Round: Technical Screen
You are asked rapid-fire ML fundamentals questions.
1. **What is overfitting?** Explain it in terms of training vs. validation performance and generalization.
2. **How can you prevent or mitigate overfitting?** List practical techniques and when you would use them.
3. **What is the difference between L1 and L2 regularization?** Compare their effect on model weights, feature selection, optimization behavior, and typical use cases.
Overview: This question evaluates understanding of overfitting, the relationship between training and validation performance and generalization, and knowledge of regularization concepts including differences between L1 and L2 penalties.
Community answers
Answer by swemleds
Overfitting occurs when model learns noises or overly specific patterns of training data, resulting in poor generalization. At the point of overfitting, training performance usually keeps improving, while that on validation data gets poorer.
It is good to have larger or more generalized train data. If not available, we get take data augmentation. Also, we can apply drop out on neural network, L1 regularization for feature selection, and early stopping when validation loss starts to increase.
L1 penalizes the absolute value of the weights, encouraging sparsity and feature selection. L2 penalizes squared weights, shrinking them toward zero but not exactly to zero. L1 is non-differentiable at zero, while L2 is smooth and easier for optimization. L1 is well-used when many features are irrelvant, and L2 is useful for general weight regularization.