Compare trees, RF, and gradient boosting
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
Explain decision trees, random forests, and gradient boosted trees. (a) List key hyperparameters for RF (n_estimators, max_features, max_depth, min_samples_leaf, bootstrap, oob_score) and for GBT (n_estimators, learning_rate, max_depth or max_leaf_nodes, subsample, min_child_weight). (b) Explain why shallow trees are typical in boosting but deeper trees can be used in RF; relate to bias–variance. (c) Define out-of-bag estimation and when it can replace a validation set. (d) For high-dimensional sparse text features, which model is preferable and why? (e) Describe how you would detect and mitigate overfitting during boosting.
Quick Answer: This question evaluates understanding of tree-based supervised learning methods—decision trees, random forests, and gradient-boosted trees—including key hyperparameters, bias–variance trade-offs, validation techniques such as out-of-bag estimation, suitability for high-dimensional sparse text features, and detection/mitigation of overfitting.