Retrieval-Augmented Generation Walkthrough: Indexing, Retrieval and Grounded Answers

Walk the interviewer through retrieval-augmented generation (RAG), step by step, from the source documents to the answer a user sees. ```hint Split the timeline Separate the work done ahead of time on the documents from the work done each time a user asks something. Then ask what the language model actually receives as input. ``` ### Clarifying Questions - How deep should I go: a product-level walkthrough, or the components (embeddings, search, prompt assembly) in some detail? - Should I ground the explanation in an example product, such as an assistant that answers customers from a company's help content? - Do you also want a comparison with alternatives such as fine-tuning a model or placing all the documents in the prompt? ### What a Strong Answer Covers - The problem RAG addresses: models lack private and recent knowledge and can state false facts confidently - The preparation phase: collecting and cleaning sources, chunking, embeddings, and a search index with metadata - The query phase: turning the question into a search, retrieving and ranking passages, assembling the prompt, and generating an answer with citations - Common failure modes, and how quality is measured separately for retrieval and for the generated answer - When RAG is the right tool versus fine-tuning or long prompts, including cost, latency, freshness and access control ### Follow-up Questions - The assistant gives a wrong answer even though the right document is in the index. How do you work out whether retrieval or generation failed? - How would you keep users from seeing content from documents they are not allowed to access? - When would you fine-tune a model instead of, or in addition to, using RAG?