Review a Compact GPT-Style Transformer Implementation
Company: Mercor
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
Overview: Review a compact decoder-only Transformer by tracing tensor shapes through causal multi-head attention, shifted next-token loss, and autoregressive generation. Examine mask correctness, padding, numerical behavior, positional schemes, context cropping, sampling, and cached-versus-uncached parity tests.