Deciding Which AI Agent Actions Need Human Approval and When to Relax That Boundary

AI agents are now capable of carrying out real actions for the people who use them. You are the product manager of an agent product and need to decide when the agent acts alone and when a person stays in the loop. Answer in two parts. ### Clarifying Questions - Is this a general assistant that acts across many apps and services, or an agent inside one product with a known, fixed set of actions? - Are the users individuals acting for themselves, or employees acting for a business, where the agent's actions affect customers or colleagues? - Do any of the actions carry regulatory or legal weight, such as moving money or handling health records? - Can actions be undone after the fact (cancelled, refunded, recalled), or are some final as soon as they happen? ### Part 1 — Where to draw the line For any given action, how do you decide whether the agent performs it by itself or a person reviews or approves it first? ```hint Not all mistakes cost the same Compare an agent that files an email into a folder with one that pays an invoice. Think about what makes a wrong action cheap or expensive to recover from, and how you would know how often the agent gets that action right. ``` #### What This Part Should Cover - The factors that place an action on the automate side or the review side - The range of oversight options between full automation and a person doing it by hand - How individual users' preferences and limits fit in ### Part 2 — Moving the line How should that boundary shift as the agent and its users gain experience, and what evidence would justify moving it in either direction? ```hint What would convince you While people still approve each action, think about what you could observe that would show the approval step has stopped adding value, and what would make you take autonomy back. ``` #### What This Part Should Cover - The evidence needed before granting more autonomy, and where it comes from - How a change in autonomy is rolled out - The signals and mechanisms for pulling autonomy back ### What a Strong Answer Covers - A framework applied to concrete example actions, not only stated as principles - Distinct levels of oversight rather than a yes-or-no choice - Measurement and governance: which metrics and thresholds move the line, and who decides - The user's experience of oversight: approval fatigue, and visibility into what the agent did - Risks specific to agents that act, such as instructions hidden in content the agent reads ### Follow-up Questions - Users approve almost every purchase the agent proposes without changing it. Do you remove the approval step? - After months of acting on its own, the agent makes one costly, public mistake on a single action type. How do you respond, and does the line move for every user? - How does your framework change for an agent acting for a whole company rather than one person?