Explain why LLMs produce hallucinations

Q: Explain why LLMs produce hallucinations

This question evaluates a candidate's understanding of the causes and mitigation of large language model hallucinations, covering competencies in probabilistic training objectives, data characteristics, model and optimization limitations, inference-time behavior, and awareness of mitigation strategies.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Question

Large language models (LLMs) are known to "hallucinate"—that is, they sometimes produce fluent, confident answers that are factually incorrect or unsupported by any source.

Explain why LLMs hallucinate. In your answer, cover:

How the standard training objective and data characteristics lead to hallucinations.
Model- and optimization-related reasons (e.g., limitations of next-token prediction, exposure bias, lack of grounding).
Inference-time factors such as decoding strategies, prompts, and distribution shift.
(Briefly) a few practical techniques used in industry to reduce hallucinations, even if they cannot be eliminated entirely.

Explain why LLMs produce hallucinations

Solution

Comments (0)

Explain why LLMs produce hallucinations

Overview

Solution

Comments (0)