Evaluates supervised learning judgment around bias, variance, overfitting, and train-test performance gaps. Strong answers diagnose leakage, split errors, distribution shift, and high variance, then propose regularization, simpler models, validation, more data, and monitoring.
##### Scenario
Model evaluation in supervised learning where training accuracy exceeds test accuracy.
##### Question
Explain the bias-variance trade-off in supervised learning. Your model performs significantly better on the training set than on the test set. What steps can you take to address this gap?
##### Hints
Relate variance to model complexity, discuss regularization, cross-validation, simpler models, more data, early stopping, ensembling, feature engineering.
Quick Answer: Evaluates supervised learning judgment around bias, variance, overfitting, and train-test performance gaps. Strong answers diagnose leakage, split errors, distribution shift, and high variance, then propose regularization, simpler models, validation, more data, and monitoring.
You are evaluating a supervised learning model and observe that training performance is much better than test performance.
Constraints & Assumptions
Treat this as a model evaluation and remediation question.
The model may be used for classification or regression, so explain concepts generally.
Separate true overfitting from data leakage, distribution shift, or evaluation errors.
Recommend practical steps, not only definitions.
Clarifying Questions to Ask Guidance
What metric is showing the train-test gap?
How were train, validation, and test sets constructed?
Is the test set from the same time period and population as training?
Are features available at prediction time, or could there be leakage?
Part 1 - Explain Bias and Variance
Explain the bias-variance trade-off in supervised learning.
What This Part Should Cover Guidance
Bias as error from overly simple assumptions.
Variance as sensitivity to training-data noise or sampling variation.
Irreducible error and why model complexity tends to reduce bias while increasing variance.
How underfitting and overfitting appear in train and test metrics.
Part 2 - Diagnose the Train-Test Gap
Your model performs significantly better on the training set than on the test set. How would you diagnose the issue?
What This Part Should Cover Guidance
Check for leakage, split mistakes, duplicate users, time leakage, label leakage, and data preprocessing leakage.
Compare train, validation, and test distributions.
Inspect learning curves, cross-validation variance, feature importance, and segment-level errors.
Part 3 - Reduce Overfitting
What practical steps can you take to address the gap?
What This Part Should Cover Guidance
Simpler model, regularization, pruning, early stopping, dropout where relevant, feature selection, and hyperparameter tuning.
More data, data augmentation, better labels, ensembling, and cross-validation.
Time-based validation and monitoring if production data drifts.
What a Strong Answer Covers Guidance
A strong answer defines bias and variance, validates that the gap is real, checks leakage and distribution shift, and proposes concrete mitigation steps tied to the model type and business use case.
Follow-up Questions Guidance
What would learning curves look like for high variance?
How would you handle overfitting in a Random Forest versus logistic regression?
What if test performance is worse only for one user segment?