Evaluating a Frontier AI Model's Safety for a Government in a 30-Day Pre-Launch Window

You work for a government agency. An AI developer gives your agency access to each new model 30 days before the model's public launch. How would you evaluate the safety of a new model within that window? The report gives only this prompt. The access, authority and resources the agency has are left open and are posed as clarifying questions below. ```hint Harms before benchmarks Decide which harms the agency is responsible for preventing before choosing any test. ``` ```hint Compared with what A score alone says little. Ask what this model can do that earlier models and ordinary tools could not. ``` ```hint The model versus the product Ask whether the version you test is the version that ships, with the same safeguards around it. ``` ### Clarifying Questions - What access does the agency get: a chat interface, API access, the model with and without its safety mitigations, or the weights? - What authority does the agency have: to advise, to require changes, or to delay the launch? - Which harms are within the agency's mandate, for example national-security risks, consumer protection, or both? - What evaluation staff, domain experts and compute can the agency use during the 30 days? - Will the developer share its own evaluation results, and can the model change between the agency's testing and the launch? ### What a Strong Answer Covers - A prioritized set of risks tied to the agency's mandate and to what is new about this model - Separate tests of the model's raw capabilities and of the safeguards meant to contain them - A mix of methods matched to risks: automated evaluations, expert red teaming, human uplift studies and agentic tests - Statistical rigor: baselines, adequate sample sizes, confidence intervals, effort spent drawing out the model's best performance, and thresholds fixed in advance - A realistic 30-day plan and a decision-oriented report: what was tested, what was found, how confident the agency is, and what it recommends ### Follow-up Questions - How would you set the thresholds that would lead you to recommend delaying a launch, and who should agree to them in advance? - Your red team finds a jailbreak that occasionally bypasses the safeguards. How do you decide whether it should block the launch? - How would you guard against underestimating the model because your prompts or tools did not draw out its full capability? - The developer updates the model a week before launch. What do you re-test?