Hand-Written Gradients: Prefix-Product Backward Pass and Stable Cross-Entropy
Company: OpenAI
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: hard
Interview Round: Onsite
Overview: Write gradients by hand: the forward and backward pass of the prefix-product operation, in linear time and without division so inputs may contain zeros, and a numerically stable softmax cross-entropy loss with its gradient in NumPy. It tests the chain rule, prefix and suffix products, and gradient checking.