Explain LLM fundamentals and trade-offs

Quick Overview

Explain LLM fundamentals and trade-offs evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Explain LLM fundamentals and trade-offs

Company: Amazon

Role: Machine Learning Engineer

Category: Machine Learning

Difficulty: hard

Interview Round: Onsite

Answer the following LLM fundamentals: How does subword tokenization (e.g., BPE) work and why is it used? Explain self-attention and the O(n^ 2) cost; discuss techniques to reduce it (e.g., sparsity, sliding windows, KV cache). Contrast pretraining, instruction tuning, and RLHF/DPO. Describe a RAG architecture, including indexing choices (BM25 vs dense, chunking, embeddings) and how retrieval quality affects generation. When do you use prompting vs fine-tuning vs adapters? Explain quantization, KV caching, and batching for low-latency inference. How do you evaluate LLMs (e.g., task-specific metrics, human eval) and mitigate hallucinations and safety risks?

Quick Answer: Explain LLM fundamentals and trade-offs evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Machine Learning/Amazon
Amazon logo
Amazon
Jul 17, 2025, 12:00 AM
hardMachine Learning EngineerOnsiteMachine Learning
9
0

Explain LLM fundamentals and trade-offs

LLM Fundamentals — Onsite Interview Task

Context: Assume a modern transformer-based LLM. Provide precise, concise explanations with examples and trade-offs.

  1. Subword tokenization (e.g., BPE): How does it work and why is it used?
  2. Self-attention: Explain the mechanism and its O(n^2) cost. Discuss techniques to reduce it (e.g., sparsity, sliding windows, KV cache).
  3. Contrast pretraining, instruction tuning, and RLHF/DPO.
  4. Describe a RAG architecture. Compare indexing choices (BM25 vs dense), chunking strategies, and embeddings. Explain how retrieval quality affects generation.
  5. When do you use prompting vs fine-tuning vs adapters (e.g., LoRA)?
  6. Low-latency inference: Explain quantization, KV caching, and batching.
  7. How do you evaluate LLMs (task-specific metrics, human eval) and mitigate hallucinations and safety risks?

Clarifying Questions to Ask Guidance

  • Clarify the task, data shape, labels, constraints, and evaluation metric.
  • State assumptions behind the math or modeling technique you choose.
  • Connect theory to practical training, debugging, and deployment implications.

What a Strong Answer Covers Guidance

  • Correct definitions and formulas where the prompt requires them.
  • A practical explanation of how the method behaves on real data.
  • Trade-offs, failure modes, diagnostics, and mitigation strategies.
  • Evaluation choices that match the product or modeling objective.

Follow-up Questions Guidance

  • How would noisy labels, class imbalance, or distribution shift affect the answer?
  • What would you monitor after deployment?
  • Which baseline would you compare against first?
Loading comments...

Browse More Questions

More Machine Learning•More Amazon•More Machine Learning Engineer•Amazon Machine Learning Engineer•Amazon Machine Learning•Machine Learning Engineer Machine Learning