Explain learning-rate fluctuation and vanishing gradients

Q: Explain learning-rate fluctuation and vanishing gradients

This is a Machine Learning interview question from Pinterest for Machine Learning Engineer roles. View the full question and solution on PracHub.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

ML Fundamentals

Answer the following conceptual questions:

Learning rate vs. training stability : Why can training metrics (loss/accuracy) fluctuate or oscillate when the learning rate is too large? What happens when it is too small?
Vanishing gradients in fully connected networks : In a deep fully connected network trained with backpropagation, are vanishing gradients more likely to affect layers closer to the input or closer to the output ? Explain why, and name common mitigations.

Explain learning-rate fluctuation and vanishing gradients

ML Fundamentals

Solution

Comments (0)