Walk through an AI agent you took from problem to production and how you evaluated it
Company: Glean
Role: Product Manager
Category: Behavioral & Leadership
Difficulty: medium
Interview Round: Other
In a product manager interview at Glean, you are asked whether you have taken an agentic AI system all the way from idea to something people actually use. If you have, walk the interviewer through it.
```hint Show it was really an agent
Expect to be pressed on whether the system chose its own steps and called tools, or was one prompt behind a chat box. Have the loop, the tools and your proof of quality ready to explain.
```
### Clarifying Questions
- Does "end to end" mean I wrote the system myself, or that I owned it as the PM from problem definition through launch while engineers built it?
- Does a side project or an internal tool count, or does the interviewer want something shipped to external users?
- Should I focus on the product decisions (user, scope, success metrics) or go deep on the architecture (retrieval, tool calls, orchestration, evaluation)?
- If I have not shipped one, should I walk through the closest thing I have built and how I would take it to production?
### What a Strong Answer Covers
- A clear user problem and why an agent that plans steps and calls tools was the right fit, rather than plain search, rules or a single prompt
- The architecture at the level a PM should own: data sources and retrieval, tools and their permissions, the planning loop, state, guardrails and human approval steps
- How quality was defined and measured: an evaluation set, task success, failure analysis, cost and latency
- Launch and iteration: who used it, what broke, what changed, and the measured outcome
- Honest ownership: what the candidate did personally versus the team, and what they would do differently
### Follow-up Questions
- How did you stop the agent from reading or acting on data the requesting user was not allowed to see?
- Walk me through one failure you found during evaluation and how you fixed it.
- If you rebuilt it today on an enterprise platform that already provides connectors and permission-aware search, what would you stop building yourself?
Overview: A Glean product manager interview question asking whether the candidate has built an agentic AI system from idea to real use, and to walk through it. It tests hands-on understanding of retrieval, tool use, permissions, guardrails and evaluation, along with honest ownership and measured results.