Designing an Evaluation Plan for a Generative AI Product Feature

How would you evaluate a generative AI system, such as an assistant or a content generator built on a large language model? Describe what you would measure and how. ```hint Split "good" apart List the separate ways this kind of output can go wrong, then decide which of them a machine can score reliably and which still need a person to judge. ``` ### Clarifying Questions - What does the system do (answer customer questions, summarize documents, draft content, write code), and who relies on its output? - Are we deciding whether to launch, or comparing a new model or prompt version against one that is already live? - Which failure would be most costly here: a confidently wrong answer, exposed sensitive data, or harmful content? - Does the system ground its answers in a knowledge source through retrieval, or does it generate freely? ### What a Strong Answer Covers - Success criteria and a ranking of failure costs specific to the system's job - A task-specific evaluation set, and how correctness, grounding, helpfulness and scope are scored on it - Where automated checks and model-based judges can be trusted, and where human review is needed - Safety, privacy and fairness testing, including adversarial inputs - Online metrics, launch thresholds, and regression checks whenever the model, prompt or data changes ### Follow-up Questions - After a prompt change, your model-based judge scores improve but user satisfaction drops. What do you investigate first? - How would you build an evaluation set for a new feature that has no labeled data yet? - Your vendor announces an updated model version. How do you decide whether to move traffic to it?