Explain overfitting, dropout, normalization, RL post-training
Company: TikTok
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
## Machine Learning fundamentals
Answer the following:
1. **What is overfitting?** How can it be mitigated in machine learning?
2. Narrowing to **deep learning**, what are common approaches to reduce overfitting?
3. **Explain dropout**:
- What does it do during training?
- Why is it considered regularization?
- How do you handle it at inference time?
4. Compare **two common normalization methods** used in deep nets (e.g., Batch Normalization vs Layer Normalization):
- What statistics do they normalize with?
- How do their behaviors differ for different batch sizes and for sequence models?
- **At deployment**, which statistics/parameters are used?
5. Describe common ways **reinforcement learning (RL)** is used in **LLM post-training** (alignment/fine-tuning after pretraining).
Quick Answer: This question evaluates understanding of model generalization and regularization concepts—specifically overfitting, dropout, normalization techniques—and the application of reinforcement learning for post-training alignment in large language models.