Explain vanishing gradients and activations
Company: Amazon
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: easy
Interview Round: Technical Screen
Explain the **vanishing gradient problem** in deep neural networks.
In your answer:
- Describe how backpropagation works at a high level and why gradients can vanish in deep networks.
- Show how the choice of **activation function** (e.g., sigmoid, tanh, ReLU) affects gradient magnitude.
- Discuss common techniques (including activation choices) to mitigate vanishing gradients.
Quick Answer: This question evaluates understanding of the vanishing gradient problem, high-level backpropagation dynamics, and the role of activation functions in gradient propagation, testing competency in neural network optimization and architecture.
