Debug a Broken Transformer
Company: OpenAI
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
Overview: This question evaluates proficiency in debugging and reconfiguring Transformer-based deep learning models, covering competencies in model internals (attention mechanisms, positional encodings), data and label pipeline alignment, tensor shaping, gradient flow, and optimizer interactions within the Machine Learning domain (sequence models / transformers). It is commonly asked because it reveals practical troubleshooting ability and architectural understanding, testing primarily practical application with necessary conceptual reasoning about attention-specific failure modes and training versus evaluation behavior.