Design an Enterprise AI Research Agent With Internal and Web Retrieval and Citations

Design an enterprise-grade AI research agent, similar in spirit to Glean-style workplace search assistants. An employee asks a complex research question. The system retrieves information from both internal company sources and external sources, synthesizes the findings, provides citations for its claims, and produces a report. Start by laying out the major components broadly before going deep on any one of them. The interviewer then drills into three areas: caching (reusing work for questions similar to earlier ones), citations, and what happens downstream when a user's permission to a source is revoked. ### Clarifying Questions - Which internal sources must be connected (documents, wikis, chat, tickets, email, code), and roughly how many employees and documents does a large customer have? - Is the result an interactive answer in seconds, or a longer report that may take minutes to produce? - Which external sources are allowed: public web search only, or licensed data providers as well? - Is this a multi-tenant cloud service, and may customer data be sent to a third-party model provider? - Can generated reports be shared with colleagues, and whose permissions apply when they are viewed? - How fresh must results be after a document changes or its sharing settings change? ### Part 1 — Core architecture: from question to cited report Design the end-to-end flow: ingestion and indexing of internal sources, access to external sources, how the agent decides what to search for, synthesis of findings, and report generation. Include the main APIs and the data model. ```hint Pipeline or planner Decide how much of the research workflow is a fixed pipeline you write in code and how much is left to the model planning its own steps and calling tools. Justify where you draw that line. ``` #### What This Part Should Cover - Components and the data flow between them, with rough scale estimates - The agent loop: planning, tool calls, stopping criteria and budgets - Permission enforcement during retrieval - Handling long-running jobs, streaming progress and partial failure ### Part 2 — Citations How do you make sure every claim in the report is attributed to a source, that each citation actually supports its claim, and that citations still make sense later? ```hint Identity of evidence Think about what identifier a piece of evidence needs so that a citation can be checked when it is generated and resolved again later. ``` #### What This Part Should Cover - How evidence is identified and carried through synthesis - Verification of citations after generation - Handling documents that change or disappear after the report is written ### Part 3 — Caching similar questions The interviewer asks how you would cache work so that a question similar to an earlier one is answered faster and more cheaply. ```hint Similar is not identical Two questions can be almost identical in wording and need different answers, and two users can ask the same question and be allowed to see different documents. ``` #### What This Part Should Cover - What is cached, at which layers, and how similarity is matched - Cache correctness under differing user permissions - Invalidation when underlying documents change ### Part 4 — Revoking permissions downstream A user loses access to a document, or is removed from a group, after that document has already been indexed, cached and cited in reports. How does the system stop exposing it? ```hint Follow the copies List every place content from one document can end up after ingestion, then decide how a revocation reaches each of them and how quickly. ``` #### What This Part Should Cover - How permission changes are detected and propagated - Where access is checked: at indexing time, at query time, and at read time - Derived artifacts such as caches, embeddings, reports and shared links ### What a Strong Answer Covers - A broad component map first, then justified depth on the areas probed - Permission-correctness treated as a hard requirement across retrieval, caching and reports - Concrete trade-offs: agentic planning versus fixed pipelines, cache hit rate versus correctness, early versus late permission checks - Reliability, cost control, security against injected instructions in retrieved content, and observability ### Follow-up Questions - A retrieved web page contains text telling the model to email a document to an outside address. How does your design prevent the agent from acting on it? - How would you evaluate report quality and citation accuracy automatically before shipping a change to the agent? - A single report now costs too much in model tokens. Where would you cut cost first without hurting quality? - How would you let a user follow up on a finished report without redoing the whole research run?