Hallucination Playbook for a Live LLM Product: Measurement, Safeguards and UX

A generative AI model is live in a product that real people use, and it sometimes hallucinates. How would you handle that? ```hint Tie the response to the harm Hallucinations cannot be removed entirely. Think about where in this product a made-up answer would do the most damage, and at which points between the user's request and the answer on screen you could step in. ``` ### Clarifying Questions - What does the product do: open-ended chat, answers drawn from a fixed set of documents, code generation, or actions taken on the user's behalf? - How costly is a wrong answer here: an annoyance, or real legal, medical or financial harm? - Do we control the model's weights, or do we call a third-party model through an API? - Are we reacting to specific reports of bad answers, or building a standing plan? ### What a Strong Answer Covers - How hallucinations are measured and tiered by severity before fixes are chosen - Measures that lower how often the model makes things up, and the limits of each - Detection of unsupported claims, both before an answer reaches the user and in production - Product and interface choices that limit the damage of hallucinations that slip through, including escalation for high-stakes cases - Metrics and trade-offs: accuracy against helpfulness and refusals, and the latency and cost of safeguards ### Follow-up Questions - After you add retrieval, the model cites real documents but sometimes misstates what they say. How do you catch and fix that? - Letting the model say "I don't know" cut hallucinations, but users now complain it refuses too often. How do you find the right balance? - A hallucinated answer becomes a public incident. What do you do right away, and what changes afterward?