Explain XGBoost depth, regularization, and dropout
Overview
This question evaluates understanding of model complexity and regularization across gradient-boosted decision trees and neural networks—covering how tree depth impacts bias/variance and computational cost, how L1/L2 and weight decay modify objectives and learned parameters, dropout behavior at inference, and distinctions between training and inference phases in Machine Learning. It is commonly asked to assess reasoning about overfitting, generalization, computational and deployment trade-offs, testing domain knowledge in supervised learning and regularization, and requires primarily conceptual understanding with practical-application considerations.
ML Conceptual Questions (Onsite)
Answer the following:
(a) Gradient-boosted decision trees: How does maximum tree depth affect bias/variance, overfitting risk, and training/inference cost? How would you choose it in practice?
(b) Neural networks: Compare L1 vs L2 regularization and weight decay — how do they modify the objective, gradients, and learned parameters?
(c) Dropout: After applying dropout during training, what should happen at inference time, and why?
(d) Training vs inference: Define and contrast these phases for ML models, including data flows, randomness, and performance considerations.
Solution
Login required