Implement NumPy neural-network layers
Company: OpenAI
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
You are given a neural-network coding task in NumPy.
Let `X` be a batch input matrix of shape `(B, d_in)`, `W` a weight matrix of shape `(d_in, d_out)`, and `b` a bias vector of shape `(d_out,)`.
1. Implement a fully connected layer for batched input: `Y = X @ W + b`.
2. Implement or explain common follow-up layers such as ReLU and softmax.
3. Explain how vector and matrix addition and multiplication should behave in NumPy, including broadcasting and shape conventions.
4. Describe how you would debug common ML implementation bugs such as transposed matrices, incorrect broadcasting, unstable softmax, and mismatched batch dimensions.
5. If a model trains poorly, explain how you would analyze the dataset for issues such as leakage, class imbalance, duplicate rows, mislabeled examples, or train/validation distribution shift.
Your answer should emphasize shape reasoning, numerical stability, and practical debugging strategy.
Quick Answer: This question evaluates competency in implementing neural-network layers and array algebra in NumPy, including matrix multiplication, broadcasting rules, activation functions, shape reasoning, numerical stability (e.g., softmax stability), and practical debugging of common ML implementation bugs.