Choosing Chunk Boundaries, Size and Overlap for Retrieval-Augmented Generation

In a retrieval-augmented generation (RAG) system, source content has to be split into pieces before it is indexed and searched. How would you decide how to split it so the system gives good answers? ```hint Work back from the question A chunk is the unit the retriever can hand to the model. Think about what a typical user question needs to see in one piece, and what goes wrong when that piece is cut in half or buried inside a much bigger one. ``` ### Clarifying Questions - What content is indexed: help-center articles, long PDFs and policy documents, code, chat transcripts, or tables? - What do users typically ask: short factual lookups, or questions that need a whole section or several documents? - Which embedding model are we using, and how much retrieved text can each answer include? - How often does the content change, and does the cost of re-indexing matter? ### What a Strong Answer Covers - How chunk size, boundaries and overlap trade retrieval precision against the context each chunk carries. - Structure-aware splitting tailored to each content type in the corpus, with useful metadata attached to every chunk. - Ways to separate what is matched from what is handed to the model, such as returning a larger parent section for a small matched chunk. - An evaluation loop that tunes the strategy with retrieval and answer-quality metrics on real queries. ### Follow-up Questions - Retrieval recall looks high, but answers still miss key details. How do you tell whether chunking is the cause? - How would you split a document whose key content sits in large tables? - How does the approach change when documents receive frequent small edits?