Explain activations, losses, and Adam

Q: Explain activations, losses, and Adam

This question evaluates understanding of neural network building blocks (layers and activation functions), comparative properties of activation/gating functions, principled selection of loss functions for different problem settings, and the internal mechanics and hyperparameters of the Adam optimizer.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Loading...

Answer the following ML fundamentals questions:

1) Neural network building blocks

What is a "layer" in a neural network, and what does it compute?
Why do we need activation functions ?

2) Activation functions deep dive

For each of the following, explain:

its mathematical form (high level is fine)
typical use cases
pros/cons and pitfalls (e.g., saturation, dead neurons, gradient flow)

Activations / gating:

Sigmoid
ReLU
SiLU (a.k.a. Swish)
SwiGLU (or GLU-style gated activations)

3) Loss functions

Given different problem settings, which loss would you choose and why?

Binary classification (possibly imbalanced)
Multi-class classification
Regression with outliers
Learning to rank / retrieval (optional)
Probabilistic forecasting (optional)

4) Optimizer

Explain how Adam works:

the moving averages it keeps
bias correction
how the parameter update is computed
what the key hyperparameters do (learning rate, $\beta_1$ , $\beta_2$ , $\epsilon$ , weight decay)

Be explicit about trade-offs, common failure modes, and practical defaults.

Explain activations, losses, and Adam

Quick Overview

1) Neural network building blocks

2) Activation functions deep dive

3) Loss functions

4) Optimizer

Solution

Comments (0)