# Design a Scalable and Safe Agentic System Design a service that accepts a user goal, lets an AI agent plan and invoke approved tools, and returns a result. Focus on three concerns: scaling many concurrent agent runs, preventing unsafe tool use, and stopping agents that repeat actions or enter an infinite loop. Define the trust boundaries and the evidence needed before an action is executed. ### Constraints & Assumptions - Model output is untrusted input, even when it looks structured. - Tools differ in risk: some are read-only, while others can spend money, modify data, or contact people. - A run may be retried after a worker or network failure. - The system must preserve an auditable record without exposing secrets in prompts or logs. ### Clarifying Questions to Ask - Which tools and data sources are available, and what permissions does each user have? - Which actions require explicit confirmation or human review? - What latency, cost, and completion guarantees matter? - Can runs be paused and resumed, and how long should state be retained? ### Solving Hints Separate orchestration from execution. Treat every proposed tool call as a request that must pass deterministic authorization, validation, budgeting, and idempotency checks outside the model. ### What a Strong Answer Covers - Durable run state, queues, worker leases, idempotency, backpressure, and fair resource budgets. - A capability-based tool gateway with authentication, authorization, schema validation, and sandboxing. - Human approval for high-impact actions and defenses against prompt injection or data exfiltration. - Loop detection using step, time, token, cost, and repeated-state limits, plus useful termination behavior. - Observability, redaction, evaluation, incident response, and explicit residual risks. ### Follow-up Questions - How would you resume safely after a worker crashes immediately after a tool succeeds? - How should untrusted content returned by one tool be presented to the model? - What metrics distinguish a difficult task from an unproductive loop?