Implement Stable Sigmoid, Softmax, and Scaled Dot-Product Attention
Company: Amazon
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
Overview: Implement numerically stable sigmoid, softmax, and single-head scaled dot-product attention using elementary arithmetic and loops. The discussion covers extreme inputs, tensor shapes, row-wise normalization, optional masks, scaling, complexity, tests, and the distinction between reference code and optimized kernels.
