Explain RF optimization and variable-importance pitfalls

Read the full interview experience this question came from →

Quick Overview

This question evaluates understanding of Random Forest regularization and feature-importance diagnostics, including recognition of biases between mean decrease impurity and permutation importance and considerations for reliable importance estimation and efficient training on large tabular datasets.

Explain RF optimization and variable-importance pitfalls

Company: Citadel

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

Explain how you would optimize and regularize a Random Forest regressor for tabular data. Cover: (1) Why classic RFs do not prune post-training and how max_depth, min_samples_leaf, and max_features control overfitting. (2) Two importance measures—mean decrease impurity vs. permutation importance—how each is computed, when they disagree, and biases (e.g., favoring high-cardinality categoricals or correlated features). (3) How to obtain reliable importances using out-of-bag estimates, repeated permutations, or conditional/permutation schemes that account for correlations. (4) Practical steps to speed training on large data (e.g., subsampling, feature bagging, warm-starting trees).

Overview: This question evaluates understanding of Random Forest regularization and feature-importance diagnostics, including recognition of biases between mean decrease impurity and permutation importance and considerations for reliable importance estimation and efficient training on large tabular datasets.

Read the full Citadel Data Scientist interview experience this question came from

|Home/Machine Learning/Citadel
Citadel logo
Citadel
Oct 13, 2025
mediumData ScientistTechnical ScreenMachine Learning
6
0

Optimize and Regularize a Random Forest Regressor for Tabular Data

Context: You are training a Random Forest (RF) regressor on tabular data and need to both regularize the model and interpret feature importance reliably, while keeping training efficient on large datasets.

Explain the following:

  1. Why classic RFs do not prune trees post-training, and how max_depth, min_samples_leaf, and max_features control overfitting.
  2. Two importance measures—mean decrease impurity (MDI) vs. permutation importance (PI): how each is computed, when they disagree, and biases (e.g., favoring high-cardinality categoricals or correlated features).
  3. How to obtain reliable importances using out-of-bag (OOB) estimates, repeated permutations, or conditional/permutation schemes that account for correlations.
  4. Practical steps to speed training on large data (e.g., subsampling, feature bagging, warm-starting trees).
Loading comments...

Browse More Questions

More Machine Learning•More Citadel•More Data Scientist•Citadel Data Scientist•Citadel Machine Learning•Data Scientist Machine Learning