Design an Agent for Internal Document Search

Quick Overview

Design an internal document-search agent with permission-aware ingestion and retrieval, grounded answers, citations, and useful abstention. Address changing access rights, document freshness, evaluation, prompt-injection defenses, latency and cost controls, observability, and large-scale serving.

Design an Agent for Internal Document Search

Company: Google

Role: Software Engineer

Category: ML System Design

Difficulty: medium

Interview Round: Technical Screen

Design an agent that answers employee questions using internal documents. Explain ingestion, permission-aware retrieval, generation, citations, evaluation, cost and latency controls, and how the service scales to a large employee population. ### Constraints & Assumptions - Source permissions can change after a document is indexed. - Answers must be grounded in documents the requesting user may read. - The system should return a useful failure when evidence is insufficient. - Do not assume a particular agent framework or exact user count beyond the stated large-scale goal. ### Clarifying Questions to Ask - Which document systems, formats, and update frequencies are in scope? - Must answers quote exact passages, perform actions, or only retrieve information? - What latency, freshness, and confidentiality requirements apply to different document classes? ### What a Strong Answer Covers - Incremental ingestion, parsing, chunking, metadata, versioning, and deletion propagation. - Identity and access control enforced during retrieval, not only during ingestion. - Hybrid retrieval, reranking, context construction, grounded generation, and source citations. - A cost and latency budget using routing, caching, smaller models, parallel retrieval, and bounded context. - Offline and online evaluation, prompt-injection defenses, auditability, scaling, and graceful degradation. ### Follow-up Questions - How would you prevent a retrieved document from instructing the agent to reveal another document? - How would you reindex a changed embedding model without losing freshness? - What would you cache without leaking results across users?

Quick Answer: Design an internal document-search agent with permission-aware ingestion and retrieval, grounded answers, citations, and useful abstention. Address changing access rights, document freshness, evaluation, prompt-injection defenses, latency and cost controls, observability, and large-scale serving.

|Home/ML System Design/Google
Google logo
Google
Aug 11, 2026
mediumSoftware EngineerTechnical ScreenML System Design
2
0

Design an agent that answers employee questions using internal documents. Explain ingestion, permission-aware retrieval, generation, citations, evaluation, cost and latency controls, and how the service scales to a large employee population.

Constraints & Assumptions

  • Source permissions can change after a document is indexed.
  • Answers must be grounded in documents the requesting user may read.
  • The system should return a useful failure when evidence is insufficient.
  • Do not assume a particular agent framework or exact user count beyond the stated large-scale goal.

Clarifying Questions to Ask Guidance

  • Which document systems, formats, and update frequencies are in scope?
  • Must answers quote exact passages, perform actions, or only retrieve information?
  • What latency, freshness, and confidentiality requirements apply to different document classes?

What a Strong Answer Covers Guidance

  • Incremental ingestion, parsing, chunking, metadata, versioning, and deletion propagation.
  • Identity and access control enforced during retrieval, not only during ingestion.
  • Hybrid retrieval, reranking, context construction, grounded generation, and source citations.
  • A cost and latency budget using routing, caching, smaller models, parallel retrieval, and bounded context.
  • Offline and online evaluation, prompt-injection defenses, auditability, scaling, and graceful degradation.

Follow-up Questions Guidance

  • How would you prevent a retrieved document from instructing the agent to reveal another document?
  • How would you reindex a changed embedding model without losing freshness?
  • What would you cache without leaking results across users?

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...