RAG system design

Design a RAG service that keeps document updates, retrieval permissions, evidence assembly, and generation consistent. Estimate index storage and measured model capacity, then test how the service behaves when evidence or dependencies are unavailable.

The preceding lesson described retrieval, context assembly, and evaluation. This lesson places those steps in a service architecture. An evaluated text model can be the generator, but a model checkpoint alone does not establish the reliability of the complete RAG service.

Start with the workload and resource budgets, then follow the document-update and query paths.

Resource estimation

Assume 100 million daily active users, ten queries each per day, and separately 100 million stored profiles. These hypothetical values establish average load; peaks and the amount of retrieved context need their own measurements.

Storage estimation

Separate standing artifacts, indexes, and retained interactions: