Adding context with retrieval-augmented generation (RAG)

Build the conceptual flow of retrieval-augmented generation: prepare documents, retrieve relevant passages, and include them in a model’s prompt. Distinguish finding useful evidence from generating a well-supported answer.

A model’s stored parameters do not automatically include recent events or your private documents. When needed facts are missing, an unsupported answer is possible. Supplying relevant evidence can help, although missing knowledge is not the only cause of incorrect output.

Retrieval-augmented generation (RAG) retrieves material from an external collection and provides it as context for generation. The open-book analogy is useful: finding a relevant passage gives the model evidence to use, but the resulting answer still needs to reflect that evidence accurately.

This lesson follows a vector-based RAG pipeline from document preparation to an augmented prompt.

Retrieval-augmented generation

When a user asks a question, retrieve candidate passages, include them with the question, and instruct the model to answer using that context. The instruction should also say what to do when the supplied evidence is insufficient.

This design offers several practical benefits: