Compare trees, RF, and gradient boosting

Q: Compare trees, RF, and gradient boosting

This question evaluates understanding of tree-based supervised learning methods—decision trees, random forests, and gradient-boosted trees—including key hyperparameters, bias–variance trade-offs, validation techniques such as out-of-bag estimation, suitability for high-dimensional sparse text features, and detection/mitigation of overfitting.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Decision Trees, Random Forests, and Gradient-Boosted Trees

You are interviewing for a Data Scientist role and are asked to compare common tree-based methods for supervised learning (classification/regression), list key hyperparameters, and reason about bias–variance and validation.

Tasks

(a) Explain decision trees, random forests (RF), and gradient-boosted trees (GBT). Then list key hyperparameters:

RF: n_estimators, max_features, max_depth, min_samples_leaf, bootstrap, oob_score.
GBT: n_estimators, learning_rate, max_depth or max_leaf_nodes, subsample, min_child_weight.

(b) Explain why shallow trees are typical in boosting but deeper trees can be used in RF. Relate to bias–variance trade-offs.

(c) Define out-of-bag (OOB) estimation and when it can replace a validation set.

(d) For high-dimensional sparse text features, which model is preferable and why?

(e) Describe how you would detect and mitigate overfitting during boosting.

Compare trees, RF, and gradient boosting

Decision Trees, Random Forests, and Gradient-Boosted Trees

Tasks

Solution

Comments (0)

Compare trees, RF, and gradient boosting

Overview

Decision Trees, Random Forests, and Gradient-Boosted Trees

Tasks

Solution

Comments (0)