Explain variance reduction in random forests

Q: Explain variance reduction in random forests

This question evaluates understanding of variance reduction in ensemble methods, the impact of inter-tree correlation on averaged predictors, and the bias-variance trade-off in random forests, framed within the Machine Learning domain for Data Scientist roles.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Loading...

Consider a random forest (or bagged ensemble) that predicts at a fixed input $x$ by averaging $B$ tree predictions:

$\hat f_B(x) = \frac{1}{B}\sum_{b=1}^B T_b(x).$

Assume each tree prediction has the same variance $\sigma^2$ and any pair of tree predictions has correlation $\rho$ .

Derive the variance of the ensemble prediction $\hat f_B(x)$ .
Explain how this connects to the variance formula for the average of correlated random variables.
Interpret the result for the cases $\rho=0$ , $\rho=1$ , and $B \to \infty$ .
Based on this result, explain why random forests use bagging and random feature subsampling, and discuss the bias-variance trade-off when making trees more random.

Explain variance reduction in random forests

Quick Overview

Solution

Comments (0)