Derive MLP shapes and explain PyTorch broadcasting

Q: Derive MLP shapes and explain PyTorch broadcasting

This question evaluates understanding of tensor shapes, linear layer forward computation, and deep-learning-framework broadcasting semantics. It is commonly asked to confirm practical ability to translate mathematical layer definitions into concrete tensor shapes and to reason about broadcasting behavior; it belongs to the Machine Learning domain and tests practical application of tensor algebra grounded in conceptual understanding.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Loading...

You are given a standard MLP layer (fully connected layer) used in deep learning.

Write the forward computation for a linear layer with bias.
Given input tensor shapes, determine output shapes and explain how bias broadcasting works in PyTorch.

Assume:

Input x has shape (B, Din) (batch size B ).
The layer has output dimension Dout .

Answer the following:

What are the shapes of weight and bias in torch.nn.Linear(Din, Dout) ?
What is the output shape?
How does this generalize if x has shape (B, T, Din) (e.g., sequence length T )?
What broadcasting rule makes bias add correctly?

Derive MLP shapes and explain PyTorch broadcasting

Quick Overview

Solution

Comments (0)