Derive MLP shapes and explain PyTorch broadcasting
Company: NVIDIA
Role: Software Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
You are given a standard MLP layer (fully connected layer) used in deep learning.
1. Write the forward computation for a linear layer with bias.
2. Given input tensor shapes, determine output shapes and explain how **bias broadcasting** works in PyTorch.
Assume:
- Input `x` has shape `(B, Din)` (batch size `B`).
- The layer has output dimension `Dout`.
Answer the following:
- What are the shapes of `weight` and `bias` in `torch.nn.Linear(Din, Dout)`?
- What is the output shape?
- How does this generalize if `x` has shape `(B, T, Din)` (e.g., sequence length `T`)?
- What broadcasting rule makes `bias` add correctly?
Quick Answer: This question evaluates understanding of tensor shapes, linear layer forward computation, and deep-learning-framework broadcasting semantics. It is commonly asked to confirm practical ability to translate mathematical layer definitions into concrete tensor shapes and to reason about broadcasting behavior; it belongs to the Machine Learning domain and tests practical application of tensor algebra grounded in conceptual understanding.