Explain learning-rate fluctuation and vanishing gradients

Q: Explain learning-rate fluctuation and vanishing gradients

This question evaluates a candidate's understanding of optimization dynamics (effects of learning rate on training stability) and backpropagation-related issues (vanishing gradients) in deep neural networks, assessing competencies in training behavior and gradient propagation within the Machine Learning domain.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

ML Fundamentals

Answer the following conceptual questions:

Learning rate vs. training stability : Why can training metrics (loss/accuracy) fluctuate or oscillate when the learning rate is too large? What happens when it is too small?
Vanishing gradients in fully connected networks : In a deep fully connected network trained with backpropagation, are vanishing gradients more likely to affect layers closer to the input or closer to the output ? Explain why, and name common mitigations.

Explain learning-rate fluctuation and vanishing gradients

ML Fundamentals

Solution

Comments (0)

Explain learning-rate fluctuation and vanishing gradients

Overview

ML Fundamentals

Solution

Comments (0)