Guardrails for an AI agent that acts for users: action tiers, approvals and permissions

As a PM at OpenAI, you own an AI assistant that can now carry out actions for its users instead of only answering them. How would you design the safeguards around those actions? ```hint Sort the actions first Not every action deserves the same protection. Look for a property of an action that tells you how much human oversight it needs, then decide which part of the system should enforce that oversight. ``` ### Clarifying Questions - Is this a consumer assistant, or an enterprise deployment where an administrator sets policy for employees? - Which actions are in scope: reading and drafting only, or also sending messages, spending money, deleting data and changing account settings? - Does the agent act through a fixed set of integrations with defined APIs, or by controlling a browser on arbitrary websites? - Do the safeguards cover only the model's behavior, or the whole experience, including permissions, the approval interface and support? ### What a Strong Answer Covers - A way to classify actions by consequence (reversibility, cost of a mistake, who else is affected) that sets how much confirmation each needs. - Where each safeguard is enforced (model, execution layer, interface) and how it holds up if the model is wrong or manipulated. - Defenses against manipulation by content the agent reads, and behavior in multi-step tasks when a step is denied or fails. - User control and transparency: what users see before and after an action, how standing permissions are granted, reviewed and revoked, and how mistakes are undone. - Metrics that balance continued delegation against error and harm rates, plus a rollout that starts with lower-stakes actions. ### Follow-up Questions - A web page the agent visits hides text telling it to forward the user's emails to an outside address. Walk through which safeguards stop it. - Users say confirmations appear constantly and they now approve without reading. How do you cut friction without losing protection? - The agent completed a purchase the user says they never wanted. Who is responsible, and what does the product do next?