Compare Gradient-Descent Optimizers and Their State
Company: Snapchat
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
List optimizers you know and explain how their update rules differ. Use plain stochastic gradient descent, momentum, an adaptive squared-gradient method, and Adam as concrete examples.
### What a Strong Answer Covers
Define the current gradient, learning rate, persistent optimizer state, and the practical reason for each modification. Explain that optimizer choice does not replace learning-rate tuning or validation.
### Follow-up Questions
- What state must be restored when resuming training from a checkpoint?
- Why might an adaptive method still fail to train a model well?
Overview: Compare SGD, momentum, AdaGrad, RMSProp, and Adam through their update state and scaling. Explain checkpoint requirements and how controlled learning-rate experiments separate optimizer behavior from model or data problems.
Read the full Snapchat Machine Learning Engineer interview experience this question came from