Add an LLM-backed recommendation API to an existing service with RAG and prompting

In a 30-minute round late in the onsite, you add an AI feature to the service you built earlier in the loop. The service can already call an open-source LLM through an API, and your task is to write a new endpoint that uses it for a recommendation use case. The interviewer is not looking for a finished, fully tested implementation. The conversation centers on retrieval-augmented generation (RAG), hallucination and, above all, how you construct the prompt and turn the model's output into something that satisfies the user story. The real service and story were not reported. For practice, assume the service manages a catalog of items (`id`, `name`, `description`, `category`, `price`) stored in a relational database, and the story is: as a shopper, I describe in plain language what I am looking for and receive a few recommended items from the catalog, each with a short reason it fits. ### Constraints and Clarifications - The LLM is reachable through an HTTP API that accepts a prompt or chat messages and returns generated text. Assume its context window cannot hold the whole catalog. - Only items that exist in the catalog may be recommended, and any item facts shown to the user, such as the price, must be correct. - Pseudocode or partial code is acceptable. Explain what you would test even if you do not write the tests. ### Clarifying Questions - How many recommendations should the endpoint return, and is it acceptable to return none when nothing in the catalog fits? - What latency is acceptable for this endpoint, and may responses be cached? - Is there user history, such as saved favorites, that recommendations may use? - Does the catalog already support keyword or vector search, or does retrieval have to be built? ### Part 1 — Design the endpoint and its flow Define the API for the story and walk through what happens on each request, from the user's text to the response. Make clear which steps are handled by ordinary code and which by the model. ```hint Not everything is the model's job Decide which parts of a recommendation have exactly one correct value, and keep those out of the model's hands. ``` #### What This Part Should Cover - The request and response contract, including limits and the empty result - How candidate items are selected before the model sees them - How the model's output is parsed, checked and combined with authoritative data - Latency, cost and failure behavior of the external model call ### Part 2 — Write the prompt Write the actual prompt you would send, both the fixed instructions and the per-request template, and explain each design choice. Show the output format you require and how your code consumes it. ```hint Write for the parser Ask what your code needs back to do its job, and make the prompt demand exactly that and nothing more. ``` #### What This Part Should Cover - A clear role, task and set of rules grounded in the supplied items - A machine-readable output format with field-level requirements - Separation of trusted instructions from user text and catalog content - Handling of the case where no item fits ### Part 3 — Retrieval, hallucination and quality Explain how retrieval-augmented generation applies here, the ways the model can still hallucinate or be manipulated even with retrieval, and how you would detect and contain each one. Then describe how you would evaluate the feature before and after launch. ```hint Trust, then verify For every field the model returns, ask how your code can confirm it before the user sees it. ``` #### What This Part Should Cover - How the retrieval step is built and what it costs - Specific hallucination and prompt-injection failure modes, each with a control - Fallback behavior when the model's output cannot be used - Offline evaluation, online evaluation and production monitoring ### What a Strong Answer Covers - A concrete request flow rather than a general description of LLMs - Prompt engineering treated as interface design: explicit rules, structured output, versioning - Grounding and validation that make it impossible to show fabricated catalog facts - Awareness of the latency, cost and availability of an external model dependency - A clear view of what to test: prompt construction, output parsing and fallbacks ### Follow-up Questions - The model sometimes returns valid JSON containing an item id that was not among the candidates. Where exactly do you catch it, and what does the user see? - How would you personalize recommendations with a user's favorites without leaking one user's data into another user's request? - Retrieval keeps missing relevant items whose descriptions use different words from the shopper's. What do you change? - When would you replace the prompt-based approach with a fine-tuned model or a classic ranking model?