Compare trees, RF, and gradient boosting

Quick Overview

This question evaluates understanding of tree-based supervised learning methods—decision trees, random forests, and gradient-boosted trees—including key hyperparameters, bias–variance trade-offs, validation techniques such as out-of-bag estimation, suitability for high-dimensional sparse text features, and detection/mitigation of overfitting.

Compare trees, RF, and gradient boosting

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Onsite

Explain decision trees, random forests, and gradient boosted trees. (a) List key hyperparameters for RF (n_estimators, max_features, max_depth, min_samples_leaf, bootstrap, oob_score) and for GBT (n_estimators, learning_rate, max_depth or max_leaf_nodes, subsample, min_child_weight). (b) Explain why shallow trees are typical in boosting but deeper trees can be used in RF; relate to bias–variance. (c) Define out-of-bag estimation and when it can replace a validation set. (d) For high-dimensional sparse text features, which model is preferable and why? (e) Describe how you would detect and mitigate overfitting during boosting.

Overview: This question evaluates understanding of tree-based supervised learning methods—decision trees, random forests, and gradient-boosted trees—including key hyperparameters, bias–variance trade-offs, validation techniques such as out-of-bag estimation, suitability for high-dimensional sparse text features, and detection/mitigation of overfitting.

|Home/Machine Learning
Oct 13, 2025
mediumData ScientistOnsiteMachine Learning
6
0

Decision Trees, Random Forests, and Gradient-Boosted Trees

You are interviewing for a Data Scientist role and are asked to compare common tree-based methods for supervised learning (classification/regression), list key hyperparameters, and reason about bias–variance and validation.

Tasks

(a) Explain decision trees, random forests (RF), and gradient-boosted trees (GBT). Then list key hyperparameters:

  • RF: n_estimators, max_features, max_depth, min_samples_leaf, bootstrap, oob_score.
  • GBT: n_estimators, learning_rate, max_depth or max_leaf_nodes, subsample, min_child_weight.

(b) Explain why shallow trees are typical in boosting but deeper trees can be used in RF. Relate to bias–variance trade-offs.

(c) Define out-of-bag (OOB) estimation and when it can replace a validation set.

(d) For high-dimensional sparse text features, which model is preferable and why?

(e) Describe how you would detect and mitigate overfitting during boosting.

Loading comments...

Browse More Questions

More Machine Learning•More Data Scientist•Data Scientist Machine Learning