Metrics and evaluation plan for an AI assistant that takes actions for users

You are the PM for a product in which an LLM carries out actions that users ask for. What would you measure to make sure it performs those actions correctly? ```hint Find where it can go wrong A request passes through several steps before anything changes in the world. List where each step could fail, and ask whether a success signal at any single step guarantees the user got what they wanted. ``` ### Clarifying Questions - What actions can the model take: low-stakes ones such as setting a reminder or playing music, or high-stakes ones such as sending messages, making payments or deleting data? - Does it act through tools and APIs, through third-party services, or by operating a user interface directly? - Does the user confirm before an action runs, or does the model act on its own? - Is the goal to evaluate the system before launch, to monitor it in production, or both? ### What a Strong Answer Covers - A definition of a correctly executed action anchored in the user's intent and the resulting outcome - Metrics for each step from understanding the request to reporting back, so failures can be located - A primary metric with guardrails, especially for actions the user never wanted - An evaluation approach that combines offline test sets, sandboxed end-to-end runs and production signals - Different bars for actions depending on risk and reversibility, and how the measurements gate launches ### Follow-up Questions - The task completion rate looks high, but support tickets about wrong actions are rising. What could explain the gap, and how do you find out? - How do you measure correctness when users silently accept whatever the model did, so you never see their true intent? - How would you decide which actions the model may take without asking the user to confirm?