ML Fundamentals — Interview Questions

Instructions

Answer the following five ML fundamentals questions. Use precise definitions, equations, and concise justifications. If a derivation is requested, show the algebra clearly.

Questions

1. Cross-entropy vs. KL divergence

• Define cross-entropy H(p, q), entropy H(p), and KL divergence KL(p || q).
• Derive the relationship H(p, q) = H(p) + KL(p || q) and explain its implications for classification loss.

2. Dropout: training/testing distribution mismatch

• Explain how dropout at training induces a distribution mismatch at test time if used naively.
• Describe practical fixes, including inverted dropout scaling and Monte Carlo (MC) dropout.
• State when you would prefer each approach in practice.

3. Normalization in large language models (LLMs)

• Justify why LLMs typically use LayerNorm instead of BatchNorm.
• Discuss implications for sequence length, micro-batching/distributed training, and training stability (e.g., pre-LN vs. post-LN).

4. Optimizers: SGD, SGD with momentum, Adam

• Write the update rules for each (include bias-corrections for Adam).
• Compare convergence behavior and generalization trade-offs.
• Give concrete guidance on when to choose each.

5. PPO (Proximal Policy Optimization)

• Explain why PPO introduces a KL-based constraint/penalty or clipping and how this stabilizes policy updates.
• Include the clipped surrogate objective and the KL-penalized objective.
• Discuss key hyperparameters, tuning strategies (e.g., target KL, clip fraction), and common failure modes.