Explain why LLMs produce hallucinations

Q: Explain why LLMs produce hallucinations

This is a Machine Learning interview question from Zillow for Machine Learning Engineer roles. View the full question and solution on PracHub.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Large language models (LLMs) are known to "hallucinate"—that is, they sometimes produce fluent, confident answers that are factually incorrect or unsupported by any source.

Explain why LLMs hallucinate. In your answer, cover:

How the standard training objective and data characteristics lead to hallucinations.
Model- and optimization-related reasons (e.g., limitations of next-token prediction, exposure bias, lack of grounding).
Inference-time factors such as decoding strategies, prompts, and distribution shift.
(Briefly) a few practical techniques used in industry to reduce hallucinations, even if they cannot be eliminated entirely.

Explain why LLMs produce hallucinations

Solution

Comments (0)