GenAI Literacy Questions: Daily AI Use, Handling Bad AI Output, and RAG Projects

Hiring managers and Bar Raisers in this company's software engineering loops are reported to ask every candidate about generative AI (GenAI) literacy. Answer the three questions below from your own experience. ### Clarifying Questions - Should I describe how my employer's rules on which AI tools may see code or data shaped the way I use them? - If I have not built a retrieval-augmented generation (RAG) system, is a course project, a prototype, or a design walkthrough acceptable? - Should the answers cover only coding, or any part of the job, such as design documents, debugging or learning a new codebase? ### Part 1 — Everyday Use How do you use GenAI tools in your day-to-day work? ```hint Specific workflows, not enthusiasm Name two or three concrete tasks where you use a tool, what you hand it, and how you check what comes back. Mention where you deliberately do not use it. ``` #### What This Part Should Cover - Concrete tasks and tools, with the input given and the output expected - How results are verified before they are trusted - Boundaries: confidential data, licensing, and tasks kept away from AI tools - Evidence of impact, stated honestly ### Part 2 — Bad AI Output Tell me about a time an AI tool gave you wrong or harmful output. How did you catch it, and what did you do? ```hint Name what caught it The strongest version of this story names the specific check that exposed the error, and the change you made so the same kind of error is caught next time. ``` #### What This Part Should Cover - The specific wrong output, and why it looked plausible - The verification step that exposed it - The fix, and the process change that prevents a repeat ### Part 3 — RAG Project Have you built a RAG project? Walk me through it: what it retrieved from, how retrieval worked, how you evaluated it, and what went wrong. ```hint Where did the bad answer come from Expect to be asked how you knew the answers were good, and how you could tell whether a wrong answer came from what was retrieved or from what was generated. ``` #### What This Part Should Cover - The problem, and why retrieval was chosen over fine-tuning or a plain prompt - The pipeline: ingestion and chunking, embedding and indexing, retrieval and ranking, prompt assembly and citation - Evaluation of retrieval quality and of answer faithfulness, with the failure cases found - Production concerns such as freshness, latency, cost, access control and prompt injection ### What a Strong Answer Covers - Concrete, first-person examples rather than general opinions about AI - A consistent verification habit, with AI output treated as a draft to be checked - Judgment about when not to use AI tools, including data-handling rules - Accurate RAG mechanics, and honesty about what was actually built ### Follow-up Questions - How do you review an AI-generated change differently from a colleague's change? - How would you detect that your RAG system had started answering from outdated documents? - A user asks the RAG system something its documents do not cover. What should it do, and how would you make it behave that way? - What would make you stop using an AI tool for a task you currently use it for?