Design cross-validation; explain bias–variance

Quick Overview

This question evaluates understanding of cross-validation methodologies and the bias–variance tradeoff, testing competency in robust model evaluation, experimental design for temporally ordered and class-imbalanced datasets, and quantitative interpretation of cross-validation error curves.

Design cross-validation; explain bias–variance

Company: Thumbtack

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Onsite

Define cross-validation rigorously and compare k-fold, stratified k-fold, leave-one-out, nested CV, and time-series rolling/blocked CV. For a dataset with temporal ordering and class imbalance, design an evaluation that avoids leakage while providing stable estimates; justify fold construction and metric choice. Explain bias–variance tradeoff quantitatively: how model complexity, training data size, and regularization shift bias and variance; how CV error curves reveal under/overfitting; and which levers (features, model class, regularization, data augmentation) you would pull when variance is high vs. when bias is high.

Quick Answer: This question evaluates understanding of cross-validation methodologies and the bias–variance tradeoff, testing competency in robust model evaluation, experimental design for temporally ordered and class-imbalanced datasets, and quantitative interpretation of cross-validation error curves.

|Home/Statistics & Math/Thumbtack
Thumbtack logo
Thumbtack
Oct 13, 2025
mediumData ScientistOnsiteStatistics & Math
7
0

Define cross-validation rigorously and compare k-fold, stratified k-fold, leave-one-out, nested CV, and time-series rolling/blocked CV. For a dataset with temporal ordering and class imbalance, design an evaluation that avoids leakage while providing stable estimates; justify fold construction and metric choice. Explain bias–variance tradeoff quantitatively: how model complexity, training data size, and regularization shift bias and variance; how CV error curves reveal under/overfitting; and which levers (features, model class, regularization, data augmentation) you would pull when variance is high vs. when bias is high.

Loading comments...