Explain Transformer and MoE Fundamentals
Company: Amazon
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
Answer the following machine learning fundamentals questions:
1. What is the difference between model training and model inference?
2. Explain the core architecture and data flow of a Transformer model.
3. What is the role of the feed-forward network inside a Transformer block?
4. How does self-attention help a model identify or emphasize important words or tokens in an input sequence?
5. Explain how the router works in a Mixture-of-Experts model.
Quick Answer: This question evaluates core machine learning fundamentals, specifically the distinction between model training and inference, the Transformer architecture and data flow including feed-forward and self-attention components, and the routing behavior in Mixture-of-Experts models, and is situated in the Machine Learning domain.
