Design a RAG system end to end

Q: Design a RAG system end to end

This question evaluates a candidate's ability to design a production-grade Retrieval‑Augmented Generation (RAG) system, testing competencies in scalable ML system architecture, embedding and vector retrieval strategies, prompt orchestration, freshness and latency engineering, security/access controls, and evaluation metrics.

Q: How do I approach ML System Design interview questions?

ML System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master ml system design interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty ML System Design question, commonly asked during Technical Screen rounds at Amazon.

Q: What role is this question designed for?

This question is commonly asked for Machine Learning Engineer candidates at Amazon during technical interviews.

Question

Design a Retrieval‑Augmented Generation (RAG) System for Enterprise Text

Context

You are building a production RAG system that answers employee questions using internal enterprise text (wikis, PDFs, tickets, emails, docs). Data is sensitive and access-controlled. Assume multi-tenant use, mixed document formats, English-first, with the following baseline constraints:

Corpus: 5–10 million pages, tens of millions of chunks.
Traffic: 200 QPS peak; target end-to-end p95 latency ≤ 2.0 s with server-streamed tokens.
Freshness: new or updated content should be searchable within 15 minutes.

Tasks

Design the system and specify:

Ingestion pipeline: chunking strategy, embedding generation, and indexing.
Retrieval strategy: vector search, hybrid retrieval, and reranking.
Prompt orchestration: how the LLM is instructed and grounded; how citations are produced.
Freshness handling: incremental updates, cache invalidation, time-aware ranking.
Latency and throughput targets with a rough budget.
Privacy and security controls for enterprise data.
Evaluation: measuring relevance and answer quality; datasets and metrics.
Reducing hallucinations: techniques across retrieval and generation.
Scale and monitoring: how you would scale, operate, and observe the system in production.

Design a RAG system end to end

Quick Overview

Design a RAG system end to end

Design a Retrieval‑Augmented Generation (RAG) System for Enterprise Text

Context

Tasks

Submit Your Answer to Earn 20XP