LLM Adaptation And PEFT

What's being tested

Interviewers are checking whether you can practically adapt large language models for production use: choose the right parameter-efficient fine-tuning (PEFT) method, design a training pipeline that fits compute and latency constraints, and evaluate tradeoffs between accuracy, cost, and operational complexity. They want to see system-level thinking (memory, throughput, serving strategy), empirical tuning (hyperparameters and validation), and monitoring/deployment practices that keep offline and online behavior aligned.

Core knowledge

PEFT taxonomy — know common classes: prompt tuning, prefix tuning, adapter layers, LoRA, BitFit, and full fine-tuning; each differs in which parameters are updated and stored versus frozen.
LoRA parameterization — represent weight updates as $W' = W + BA$ with $B∈R^{d×r}, A∈R^{r×k}$ ; storage scales with $2·r·(d+k)$ , so small $r$ (e.g., 4–64) yields large savings.
Quantization fundamentals — 8-bit and 4-bit inference reduce memory and bandwidth; bitsandbytes and transformers enable 8/4-bit weights and optimizer state; quantization may increase perplexity and require calibration.
QLoRA pattern — combine 4-bit quantization with LoRA-style updates so full model stays quantized on GPU while low-rank adapters are trained in FP16/BF16; reduces memory enough to fine-tune 30B+ models on single GPUs.
Optimization & memory tricks — use fp16/bf16, gradient accumulation, mixed precision, DeepSpeed ZeRO stages (especially ZeRO-3) or accelerate to distribute state; tradeoff: ZeRO reduces memory at cost of all-reduce and latency.
Hyperparameter heuristics for PEFT — use smaller learning rates (e.g., $1e-4–5e-5$ ) and fewer warmup steps than full-tuning; batch size impacts stability—use gradient accumulation to simulate larger batches.
Evaluation metrics & validation — for instruction-tuning track task-specific metrics (accuracy, F1), plus perplexity and calibration (confidence vs accuracy); for conversational/instruction models include automated reward-model wins and small-scale human evals.
Serving strategies — adapter-merge (merge LoRA into base) for low-latency single-tenant inference; on-the-fly adapter injection for multi-tenant; cache merged weights in fast storage (NVMe) to avoid repeated merges.
Operational monitoring — monitor latency $p_{50}/p_{95}/p_{99}$ , GPU memory pressure, token throughput, distributional drift (embedding distance, KL divergence, change in perplexity) and user-facing metrics (error rate, fallback rate).
Model lifecycle & storage — store many small adapter artifacts instead of multiple full checkpoints; version adapters with metadata (base model hash, tokenizer, quantization bits, LoRA rank, dataset snapshot).
Failure modes & safety — PEFT can underfit domain shifts or maintain old behavior (catastrophic forgetting less likely than full fine-tuning); watch hallucinations and calibration shifts after adaptation.
Cost/benefit calculus — estimate cost by memory footprints and GPU-hours: adapting with LoRA + 4-bit quantization often reduces GPU memory ~3–10× compared to full FP16 fine-tuning, enabling cheaper experiments and faster iteration.

Worked example

Problem framing: "Adapt a 34B LLM to a new enterprise domain with 10k labeled pairs and tight latency." First ask clarifying questions: required latency and throughput, whether multi-tenant adapters are needed, available GPU memory, and evaluation success metrics. Skeleton plan: (1) choose QLoRA + LoRA so the base model remains 4-bit quantized and only low-rank adapters are trained; (2) preprocess and split data for SFT, hold out a validation set and a small human-eval set; (3) pick LoRA rank $r$ (start 8–16), bf16 training, gradient accumulation, and DeepSpeed/accelerate for training stability; (4) evaluate with task metrics, perplexity, and a small instruction-following human check. Tradeoff flagged: increasing $r$ raises adaptation capacity but increases latency and storage per adapter—start small and scale $r$ based on validation gains. Closing: "If I had more time, I'd run a grid over $r$ and learning rate, do adapter-merge experiments for serving, and run small-scale RLHF or reward-model tuning to align outputs to business preferences."

A second angle

Consider a different constraint: "support thousands of customers each needing a personalized adapter, with strict per-request latency." The same PEFT concept applies but operational constraints dominate. Instead of merging adapters into full models for each customer (storage and load time explosion), use a runtime adapter-injection server that caches the most-active merged weights in GPU memory and serves others from an on-disk merged store with async warm-up. Batch requests by adapter id to amortize merge overhead. Here the adaptation design prioritizes adapter storage efficiency (LoRA with small $r$ , compressed adapters) and a cache eviction policy based on QPS per tenant rather than maximizing per-adapter accuracy.

Common pitfalls

Pitfall: Underestimating memory/perf constraints — assume LoRA automatically fits your device; you must account for quantized base weights, optimizer state, and activation memory, or you'll OOM during training.

Pitfall: Using full-finetuning hyperparameters — PEFT changes effective parameter scale; use lower learning rates and monitor validation loss closely to avoid divergence or catastrophic overfitting to small datasets.

Pitfall: Ignoring serving complexity — proposing many small adapters without a serving plan leads to high cold-start latency and increased operational cost; articulate caching, merge-on-deploy, or multi-tenant injection strategies.

Connections

Interviewers may pivot to model compression & quantization (practical tradeoffs and tools), training infra (ZeRO, pipeline parallelism), or evaluation & monitoring (automated vs human evaluation, calibration, and drift detection).