Design an Agent for Internal Document Search
Company: Google
Role: Software Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Technical Screen
Design an agent that answers employee questions using internal documents. Explain ingestion, permission-aware retrieval, generation, citations, evaluation, cost and latency controls, and how the service scales to a large employee population.
### Constraints & Assumptions
- Source permissions can change after a document is indexed.
- Answers must be grounded in documents the requesting user may read.
- The system should return a useful failure when evidence is insufficient.
- Do not assume a particular agent framework or exact user count beyond the stated large-scale goal.
### Clarifying Questions to Ask
- Which document systems, formats, and update frequencies are in scope?
- Must answers quote exact passages, perform actions, or only retrieve information?
- What latency, freshness, and confidentiality requirements apply to different document classes?
### What a Strong Answer Covers
- Incremental ingestion, parsing, chunking, metadata, versioning, and deletion propagation.
- Identity and access control enforced during retrieval, not only during ingestion.
- Hybrid retrieval, reranking, context construction, grounded generation, and source citations.
- A cost and latency budget using routing, caching, smaller models, parallel retrieval, and bounded context.
- Offline and online evaluation, prompt-injection defenses, auditability, scaling, and graceful degradation.
### Follow-up Questions
- How would you prevent a retrieved document from instructing the agent to reveal another document?
- How would you reindex a changed embedding model without losing freshness?
- What would you cache without leaking results across users?
Quick Answer: Design an internal document-search agent with permission-aware ingestion and retrieval, grounded answers, citations, and useful abstention. Address changing access rights, document freshness, evaluation, prompt-injection defenses, latency and cost controls, observability, and large-scale serving.