You are interviewing for an AI-focused Software Engineer role at a company building products on top of large language models. During the screen, the interviewer asks: **"How would you prevent — or at least significantly reduce — hallucinations in an LLM-based system?"** Walk through the causes of hallucination and the concrete techniques you would apply across the stack (training, inference, retrieval/grounding, verification, product/UX, and evaluation) to keep the system from confidently stating things that are false or unsupported. ```hint Where to start Don't jump straight to a single fix. First define what "hallucination" means for the product (unsupported claims vs. factually false claims vs. fabricated citations), then organize mitigations by **where in the lifecycle** they act: before generation (data/training), during generation (grounding + decoding), and after generation (verification + UX). ``` ```hint The highest-leverage technique For most production systems the biggest single win is **grounding**: retrieval-augmented generation (RAG) or tool use that puts authoritative context in the prompt, plus instructions (and training) that make the model answer *only* from that context and abstain otherwise. Then think about how you'd *verify* the output against the retrieved sources. ``` ### Constraints & Assumptions - Assume a production LLM-based application (e.g., a question-answering assistant or agent), not a research prototype. - You may or may not control model pretraining; assume you can fine-tune, prompt, and build infrastructure around the model. - Latency and cost matter: mitigations that multiply inference cost need justification. - "Prevent" should be interpreted honestly — hallucination can be reduced and contained, not eliminated with certainty. ### Clarifying Questions to Ask - What kind of product is this — open-domain chat, domain-specific Q&A over private data, code generation, or an autonomous agent? The dominant failure mode differs. - How costly is a hallucination here? Is this a casual assistant, or a high-stakes domain (medical, legal, financial) where a wrong answer causes real harm? - Do we control the model (can fine-tune / RLHF) or are we consuming a third-party API where only prompting and system-level defenses are available? - Is there an authoritative source of truth we can ground against (documents, database, APIs), or is the model expected to answer from parametric knowledge? - What latency and cost budget do we have per request (affects whether multi-pass verification or self-consistency is feasible)? ### What a Strong Answer Covers ```premium-lock What a Strong Answer Covers ``` ### Follow-up Questions - Your RAG system still hallucinates: the answer contradicts the retrieved passages. What are the likely causes and how do you fix each one? - How would you build an automated evaluation pipeline to measure hallucination rate on every model or prompt change, and what would you use as ground truth? - When is it better for the model to say "I don't know," and how do you actually train or prompt a model to abstain without it becoming uselessly evasive? - How do these mitigations change for an agent that takes actions (calls APIs, writes code) rather than one that just produces text?