Explain learning-rate fluctuation and vanishing gradients
Company: Pinterest
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: easy
Interview Round: Technical Screen
### ML Fundamentals
Answer the following conceptual questions:
1. **Learning rate vs. training stability**: Why can training metrics (loss/accuracy) **fluctuate or oscillate** when the learning rate is too large? What happens when it is too small?
2. **Vanishing gradients in fully connected networks**: In a deep fully connected network trained with backpropagation, are **vanishing gradients** more likely to affect layers closer to the **input** or closer to the **output**? Explain why, and name common mitigations.
Quick Answer: This question evaluates a candidate's understanding of optimization dynamics (effects of learning rate on training stability) and backpropagation-related issues (vanishing gradients) in deep neural networks, assessing competencies in training behavior and gradient propagation within the Machine Learning domain.