Design LLM search handling long token inputs

You are asked to design an LLM-powered search system that lets users query a large corpus of documents (e.g., internal wikis, PDFs, logs, and web pages) and receive natural-language answers.

A key challenge is that both documents and user queries can be very long, often exceeding the context window (maximum token length) of the underlying large language model (LLM). For example, a user might paste multiple pages of logs or a long contract as part of their query.

Design the system with a focus on:

1. Overall architecture
   • How documents are stored and indexed.
   • How search queries are processed.
   • How the LLM is used to generate final answers.
2. Handling large token length / context limits
   • How to handle very long documents that do not fit into the LLM context.
   • How to handle very long queries (e.g., multi-page text pasted by the user).
   • How to avoid blowing past the context window while still providing high-quality, relevant answers.
3. Additional considerations
   • Latency and cost: how you keep response times reasonable and control token usage.
   • Quality: how you keep retrieved content relevant and avoid missing important context when chunking or summarizing.
   • Any caching or optimizations you would introduce.

Describe your design in detail:

• Draw or describe the main components and data flow (ingestion, indexing, retrieval, LLM interaction, etc.).
• Explain at least 2–3 concrete strategies for dealing with large token length/context limits, and how they fit into your architecture.
• Call out trade-offs between different design choices.