Assemble a RAG Pipeline From Provided Python Interfaces and Explain Retrieval

In this round the interviewer hands you a set of Python interfaces, some of them already implemented, and asks you to assemble a retrieval-augmented generation (RAG) pipeline from them. Pseudocode is acceptable. The pieces combine several technologies, so expect parts of the stack to be unfamiliar: ask clarifying questions, take the interviewer's guidance, and keep explaining your reasoning. Along the way the interviewer asks conceptual questions about why the pipeline works. The exact interfaces from the interview are not reproduced. For practice, assume you are given the following, where `Embedder`, `VectorStore` and `LLM` are already implemented and you write the code that wires them together: ```python from dataclasses import dataclass @dataclass class Document: doc_id: str text: str source: str @dataclass class Hit: chunk_id: str score: float # similarity score computed by the store metadata: dict # whatever was passed to add() for this chunk class Embedder: def embed(self, texts: list[str]) -> list[list[float]]: ... class VectorStore: def add(self, ids: list[str], vectors: list[list[float]], metadata: list[dict]) -> None: ... def query(self, vector: list[float], k: int) -> list[Hit]: ... class LLM: def generate(self, prompt: str) -> str: ... ``` ### Clarifying Questions - Which pieces are already implemented and which am I expected to write, for example chunking? - Does `VectorStore.query` score by cosine similarity, dot product or distance, and does `Embedder` return normalized vectors? - How large is the corpus and how often does it change: is indexing a one-time batch or continuous? - Should answers cite their sources, and what should happen when nothing relevant is found? - Are there limits on the model's context window or on the latency of one answer? ### Part 1 — Assemble the pipeline Write the indexing path that turns a list of `Document` objects into searchable entries, and the query path that answers a user question using retrieved context. ```hint Two paths, one embedding space Separate what runs once per document from what runs once per question, and check that both sides produce vectors that can be compared with each other. ``` #### What This Part Should Cover - Chunking with stable IDs and metadata that let an answer cite its source - Batched embedding and indexing on the write path - Query embedding, top-k retrieval, and a prompt that grounds the model in the retrieved text - Behavior when retrieval returns nothing useful ### Part 2 — Explain the retrieval mechanics Explain why the pipeline needs embeddings at all, how retrieval finds relevant chunks, and why cosine similarity is a reasonable way to compare the query vector with the stored vectors. ```hint Direction versus length Think about what the embedding model was trained to place close together, and what the length of a vector does or does not tell you about meaning. ``` #### What This Part Should Cover - What an embedding represents and what keyword matching alone would miss - Exact versus approximate nearest-neighbor search, and the trade-off between them - The cosine similarity formula, its relationship to dot product and Euclidean distance, and its limits ### Part 3 — Decide when RAG is the right tool When is RAG the right approach, and when would you choose something else, such as fine-tuning, putting the whole corpus in the prompt, or a plain search or database query? ```hint Classify the request Ask whether the request needs facts the model does not have, a change in how the model behaves, or a computation over many records. ``` #### What This Part Should Cover - Signals that favor RAG: private, large or frequently changing knowledge, and a need for citations - Cases where fine-tuning, long context or structured queries win instead - The cost, latency and maintenance implications of each choice ### What a Strong Answer Covers - Working, readable pipeline code that respects the given interfaces rather than inventing new ones - Correct explanations of embeddings, nearest-neighbor retrieval and cosine similarity - A grounded prompt with citations and an explicit "not found" path - Awareness of evaluation: retrieval quality measured separately from answer quality - Recovery under unfamiliar tooling: asking focused questions and using hints without stalling ### Follow-up Questions - Your corpus is full of product codes and error identifiers that users type exactly. How does that change retrieval? - You switch to a new embedding model. What must happen to the existing index, and why? - How would you measure whether a change to chunk size made the system better? - The model ignores the retrieved context and answers from memory. How do you detect and reduce that?