Derive MLP shapes and explain PyTorch broadcasting

Q: Derive MLP shapes and explain PyTorch broadcasting

This is a Machine Learning interview question from NVIDIA for Software Engineer roles. View the full question and solution on PracHub.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Loading...

You are given a standard MLP layer (fully connected layer) used in deep learning.

Write the forward computation for a linear layer with bias.
Given input tensor shapes, determine output shapes and explain how bias broadcasting works in PyTorch.

Assume:

Input x has shape (B, Din) (batch size B ).
The layer has output dimension Dout .

Answer the following:

What are the shapes of weight and bias in torch.nn.Linear(Din, Dout) ?
What is the output shape?
How does this generalize if x has shape (B, T, Din) (e.g., sequence length T )?
What broadcasting rule makes bias add correctly?

Derive MLP shapes and explain PyTorch broadcasting

Comments (0)