Explain Transformer, GPT vs BERT, and PR metrics
Overview
This question evaluates understanding of modern NLP model architecture and evaluation metrics, specifically Transformer block components, key distinctions between GPT and BERT including pretraining objectives and usage, and precision/recall interpretation, testing competencies in deep learning architecture knowledge, model selection reasoning, and metric-based evaluation within the Software Engineering Fundamentals domain for a Machine Learning Engineer role. It is commonly asked because it probes both conceptual understanding of architectures and pretraining paradigms and practical application of performance trade-offs—assessing conceptual understanding alongside practical application in threshold-based precision versus recall decisions.
Answer the following conceptual questions:
-
Transformer architecture
- Describe the main components of a Transformer block and what each part does.
-
GPT vs BERT
- Explain the key differences in architecture usage and pretraining objectives.
- When would you prefer one over the other?
-
Precision and recall
- Define precision and recall.
- Give an example of how changing a threshold can trade off precision vs recall.
- Mention at least one scenario where you prioritize precision and one where you prioritize recall.