System Design: LLM-Powered Customer Support Bot

A support bot must handle questions that do not match a predefined script while preserving the rules of the underlying support process. Some requests need a document lookup, some need an account API, and others need a human agent.

We will design a bot that uses a language model to interpret requests and draft responses, retrieval to supply approved support knowledge, and narrow APIs to obtain live account state. Retrieval does not guarantee that the source is current or that the answer follows it. The original RAG paper reports improvements on evaluated tasks; our support application still needs its own tests and an explicit fallback when evidence is missing.

This lesson establishes three things: functional requirements, nonfunctional requirements, and resource estimation for designing an LLM-powered customer support bot.

Let’s start with the functional requirements.

Functional requirements

The following functional requirements define the system’s core behavior: