## LLM Foundations: Architecture, Adaptation, and Steering You are a software engineer interviewing for a team that is integrating a large language model (LLM) into a product. The interviewer is not testing whether you can derive backpropagation by hand; they want to confirm you have an accurate working mental model of how modern LLMs are built, how they are adapted to a task, and how their behavior is steered at inference time. The conversation is conceptual — no coding is required, but you are expected to reason precisely about trade-offs. ### Constraints & Assumptions - Assume modern **decoder-only** Transformer LLMs (the GPT/Llama family style), not encoder-decoder translation models, unless you call out the difference yourself. - This is a whiteboard / discussion question: explain mechanisms and trade-offs rather than write training code. - "Adaptation" here covers everything you might do to make a general base model useful for a specific product: post-training, fine-tuning, retrieval, and prompting. - Assume a realistic product setting: there is a latency and cost budget, only a modest amount of labeled in-domain data, and the option to either call a hosted model or host an open-weight model yourself. ### Clarifying Questions to Ask - Are we constrained to a hosted API model (weights frozen, prompt-only control) or can we host and fine-tune an open-weight model? - How much labeled in-domain data do we have, and how is its quality? - What are the latency, throughput, and cost-per-request budgets? - What does "good" mean for this product — factual accuracy, style/tone, structured output, safety, or all of these? - Does the task need access to private or frequently-changing knowledge that was not in the model's pretraining data? ### Part 1 — Transformer fundamentals Walk the interviewer through how a decoder-only Transformer turns a sequence of input tokens into a next-token prediction. Cover the self-attention mechanism (the query/key/value projections and how attention weights are formed), why multiple attention heads are used, how the model knows token order, and the role of the feed-forward layers together with residual connections and normalization. Finally, explain why the attention computation scales **quadratically** with sequence length and what practical consequence that has. ```hint Where to start Each token position produces three projections of its embedding. Name them, and describe how one token decides how much to "read" from every other token using two of those three. ``` ```hint Order and depth Attention itself is permutation-agnostic, so something extra must inject position. And a block is more than attention — what does the per-position feed-forward sublayer add, and why do residual connections matter for training deep stacks? ``` ```hint Cost Count the pairwise interactions among $n$ tokens when every position attends to every other. That count, times the per-pair work, is the bottleneck. ``` #### What This Part Should Cover ```premium-lock What This Part Should Cover ``` ### Part 2 — Post-training and fine-tuning A team hands you a **pretrained base** model. Explain what "post-training" means and why a raw base model is usually a poor product assistant despite having absorbed huge amounts of text. Compare **supervised fine-tuning (SFT)** with **preference optimization** (RLHF / DPO). Then compare **full fine-tuning** against **parameter-efficient fine-tuning** (LoRA / adapters). Close with a decision rule: when is fine-tuning actually the right tool versus relying on prompting or retrieval? ```hint Framing Pretraining optimizes one objective: predict the next token over web-scale text. Ask what behavior that objective does and does not instill for a helpful, instruction-following assistant. ``` ```hint What PEFT actually trains LoRA does not update the original weight matrices. Think about what small set of parameters it injects and why that slashes optimizer-state memory and lets you keep one base model with many swappable adapters. ``` #### What This Part Should Cover ```premium-lock What This Part Should Cover ``` ### Part 3 — Prompt engineering and the adaptation decision Now suppose you **cannot** change the model weights at all (hosted API, frozen model). Explain how you still steer behavior through the input context. Cover zero-shot vs few-shot prompting, chain-of-thought / step-by-step prompting, system prompts and role framing, eliciting structured output, and retrieval-augmented generation (RAG). Then give a crisp framework for choosing among **prompting**, **RAG**, and **fine-tuning** for a new task. ```hint The only lever With frozen weights you control exactly one thing: the tokens you feed in. Enumerate what you can place in that context to demonstrate the task, supply missing facts, or constrain the output shape. ``` #### What This Part Should Cover ```premium-lock What This Part Should Cover ``` ### What a Strong Answer Covers ```premium-lock What a Strong Answer Covers ``` ### Follow-up Questions - How does the context-window limit connect to the quadratic attention cost from Part 1, and what techniques reduce it (KV caching, FlashAttention, sparse or linear attention, sliding-window attention)? - What is catastrophic forgetting, and how does it influence the choice between full fine-tuning and LoRA? - In RLHF, what is the reward model and why does DPO claim to simplify the pipeline? - How would you actually *evaluate* whether a fine-tune or a prompt change improved the product, beyond eyeballing a few outputs? - The model confidently states a false fact. Walk through how you would diagnose whether prompting, RAG, or fine-tuning is the right fix.