Identify What Does Not Fix Vanishing Gradients

# Identify What Does Not Fix Vanishing Gradients Which change is not a reliable remedy for vanishing gradients in a deep neural network: adding skip connections, using an activation such as ReLU, raising the learning rate, or adding batch normalization? Explain the mechanism behind your choice and the limitations of the other interventions. ### What a Strong Answer Covers - Gradient propagation through a product of layer derivatives. - Why increasing the optimizer step size does not repair a vanishing gradient signal. - How skip connections, activation choice, and normalization can help without guaranteeing a cure. ```hint Separate signal from step size The gradient is computed before the optimizer multiplies it by the learning rate. ``` ### Follow-up Questions - Can ReLU units still stop transmitting gradients? - Why can residual connections help very deep models?