Explain overfitting, imbalance, undersampling, and attention heads

Q: Explain overfitting, imbalance, undersampling, and attention heads

This question evaluates a candidate's competency in diagnosing and mitigating overfitting, handling class imbalance including undersampling strategies, and understanding transformer attention heads and their effects on model capacity and computation.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty Machine Learning question, commonly asked during Technical Screen rounds at TikTok.

Q: What role is this question designed for?

This question is commonly asked for Machine Learning Engineer candidates at TikTok during technical interviews.

Question

Context

You are designing and evaluating production machine learning models, with emphasis on classification, reliability, and efficient architectures. Answer the following multi-part question.

Part 1 — Diagnose and Mitigate Overfitting

Explain how to diagnose overfitting and discuss mitigation techniques, including:

Regularization: L1, L2 (weight decay)
Data augmentation (and label-preserving transformations)
Early stopping and learning-rate schedules
Dropout and related stochastic regularizers
Architecture changes (capacity, inductive bias, normalization)
Cross-validation
Proper validation strategies (avoiding leakage; stratified/group/time-based splits) For each technique, explain trade-offs.

Part 2 — Handle Class Imbalance

Discuss approaches to class imbalance:

Undersampling vs. oversampling (random; SMOTE and variants)
Class weighting, focal loss, and threshold selection
When each method is appropriate and how to evaluate with precision/recall, PR-AUC, ROC-AUC, and calibration

Detail common undersampling strategies and their impact on bias/variance and minority-class recall:

Random undersampling
Tomek links
Edited Nearest Neighbors (ENN)
NearMiss variants
Cluster-centroid methods

Part 3 — Attention Heads in Transformers

Define an attention head and explain:

Queries, keys, values
How multi-head attention splits representation space
What multiple heads can capture
How head count affects capacity, compute, and inductive bias

Explain overfitting, imbalance, undersampling, and attention heads

Quick Overview

Context

Part 1 — Diagnose and Mitigate Overfitting

Part 2 — Handle Class Imbalance

Part 3 — Attention Heads in Transformers

Solution

Comments (0)