Explain Transformer and Fine-Tuning Basics
Company: Lyft
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
You are interviewing for an AI-focused engineering internship. Explain the following:
1. What is the difference between a transformer model and an embedding model? Describe what each one produces, how they are commonly trained, and when you would use one instead of the other.
2. What is the difference between pretraining and fine-tuning? Explain why pretraining is useful, what changes during fine-tuning, and what trade-offs or failure modes you should watch for.
Quick Answer: This question evaluates understanding of transformer architectures, embedding models, and transfer learning concepts such as pretraining and fine-tuning, testing competence in representation learning and model adaptation.