Quick Overview

This coding question tests a machine learning engineer's understanding of Transformer internals by requiring a from-scratch NumPy implementation of multi-head scaled dot-product attention. It evaluates mastery of linear projections, head splitting, numerically stable softmax, and causal masking — key competencies for ML roles involving large language models.

Implement Multi-Head Attention from Scratch in NumPy

Company: Amazon

Role: Machine Learning Engineer

Category: Coding & Algorithms

Difficulty: medium

Interview Round: Onsite

Quick Answer: This coding question tests a machine learning engineer's understanding of Transformer internals by requiring a from-scratch NumPy implementation of multi-head scaled dot-product attention. It evaluates mastery of linear projections, head splitting, numerically stable softmax, and causal masking — key competencies for ML roles involving large language models.

Loading…