Walk Through a Generative-AI Product Project and a Time It Did Not Go to Plan

This is a 30-minute hiring-manager screen for a software engineer role on a team that builds generative-AI product features from zero to one, working with a research team across the whole life cycle: choosing a model, launching, measuring and improving. There is no coding. The interviewer asks about one of your projects, its goal, scope and impact, and then asks about something that did not go to plan. ### Clarifying Questions - Should the project feature a generative-AI component you shipped, or may it be any project you led? - With 30 minutes for both questions and your own questions, how long should the first walkthrough run before the interviewer starts probing? ### Part 1 — Walk through a project: goal, scope and impact Choose a project, ideally one where you shipped a generative-AI feature, and walk the interviewer through its goal, its scope, your role and decisions, and its impact. Expect the interviewer to probe the AI parts. ```hint Scope before detail Before describing any code, say how big the project was, what you owned and what you deliberately left out; for this role the interviewer is listening for judgment. ``` #### Clarifying Questions for this Part - Does the interviewer want depth on the model and evaluation decisions, or on product impact and the work with the research team? #### What This Part Should Cover - Goal, scope and your role stated up front, with the calls you made and the options you gave up - The AI-specific decisions: why that model, how quality was measured, how cost was controlled, and how bad outputs were caught before they reached users - Impact measured against a baseline, and what you would do next ### Part 2 — Something that did not go to plan On that project, or another one if it fits better, describe something that did not go to plan. ```hint Spend the time on the recovery Choose a miss whose cause you understand, keep the setup short, and spend most of the answer on what you did once you noticed it and what changed afterwards. ``` #### What This Part Should Cover - A real problem, how and when it surfaced, and your share of the cause - Your actions in order, with the trade-offs behind them - The result, what you learned, and what you changed so it would not happen again ### What a Strong Answer Covers - A project that fits a generative-AI product role, told scope first, with judgment and trade-offs rather than a list of the code you wrote - Defensible choices on model selection, evaluation, cost and fallbacks, beyond "we called an API" - Impact with an honest baseline, and clarity about which decisions were yours and which were the team's - A complete STAR account of the setback: ownership without blame, specific actions, a measured result and a lasting change - Concise first passes that leave room for the interviewer's probes inside a 30-minute slot ### Follow-up Questions - Why that model rather than a larger or smaller one, or a non-generative approach? - How did your offline evaluation compare with what users experienced after launch, and what did you do when the two disagreed? - What happened when the model produced a wrong or unsafe output, and what stopped it from reaching the user? - Looking back at the setback, what would you put in place on day one of your next project?