Implement Masked Multi-Head Self-Attention

Implement Masked Multi-Head Self-Attention

Company: Apple

Role: Machine Learning Engineer

Category: Machine Learning

Difficulty: easy

Interview Round: Technical Screen

Overview: This question evaluates implementation and conceptual understanding of masked multi-head self-attention, covering scaled dot-product attention, separate linear projections for queries/keys/values, head-wise tensor reshaping, and the construction and application of padding and causal masks.