Design a RAG-Based Agent System
Company: Bytedance
Role: Software Engineer
Category: ML System Design
Interview Round: Technical Screen
Design and explain a retrieval-augmented generation system that supports an LLM-based agent.
Cover the following topics:
1. How would you build a RAG pipeline end to end?
2. How would you use frameworks such as LangGraph or LangChain to organize the workflow?
3. What is the difference between an agent tool and an MCP-style external tool integration?
4. How would the agent support memory across conversations?
5. If a user has a very long conversation that exceeds the model context window, for example more than 245,000 tokens, how would you prevent the system from failing or producing poor answers?
6. How would you evaluate the quality, latency, and reliability of the system?
Quick Answer: This question evaluates a candidate's competency in designing retrieval-augmented generation (RAG) systems and LLM-based agents, covering retrieval pipelines, tool integration patterns, conversational memory, long-context management, and system evaluation metrics.