Explain overfitting, imbalance, undersampling, and attention heads
Company: TikTok
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: hard
Interview Round: Technical Screen
How do you diagnose and mitigate overfitting in machine learning models? Discuss techniques such as regularization (L1/L
2), data augmentation, early stopping, dropout, architecture changes, cross-validation, and appropriate validation strategies; explain trade-offs for each. How do you handle class imbalance? Compare undersampling, oversampling (including SMOTE/variants), class weighting, focal loss, and thresholding; specify when each is appropriate and how to evaluate with metrics like precision/recall, PR-AUC, ROC-AUC, and calibration. Detail common undersampling strategies—random undersampling, Tomek links, Edited Nearest Neighbors (ENN), NearMiss variants, and cluster-centroid methods—and discuss their impact on bias/variance and minority class recall. What is an attention head? Explain queries, keys, and values; how multi-head attention splits representation space; what multiple heads capture; and how head count affects capacity, compute, and inductive bias.
Quick Answer: This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Explain overfitting, imbalance, undersampling, and attention heads states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.