Classify and cluster business records into industries using LLMs

You have a large collection of business records with fields such as the business name and address. Design a system that classifies or clusters these businesses into industries such as Banking, Tech, Healthcare and Pharma, using large language models (LLMs). The interviewer posed it as an open-ended system design question, so the scale, the inputs available and the quality bar are yours to clarify. ```hint How much does a record say Consider how much a name and an address actually reveal about an industry, which records a cheap method can label confidently, and which ones need more context before any model can decide. ``` ```hint Trusting the labels Ask how you would measure whether the LLM's labels are right, and what the system should do when the model is unsure or answers with a label outside your list. ``` ### Clarifying Questions - Is the list of industries fixed in advance (classification), should the system discover groups on its own (clustering), or both? - How many records are there, how many new ones arrive per day, and is this a one-time backfill or also an ongoing service? - Which fields exist besides name and address, and may the system use outside data such as the business's website? - Can a business belong to more than one industry, and how fine-grained should the industries be? - Are any records already labeled, and what accuracy, cost and latency targets apply? - May the records be sent to an external LLM provider, or must the model run in-house? ### What a Strong Answer Covers - Problem framing: fixed taxonomy versus discovered clusters, single versus multiple labels, batch versus online - Record cleaning, deduplication and enrichment when name and address are too thin to decide - A pipeline that sends only the records that need it to the LLM, with output constrained to the taxonomy, a confidence signal and an abstain option - A clustering path for unknown industries, with clusters named and reviewed before they become labels - Evaluation on a labeled sample with per-industry metrics, and a human review loop that feeds corrections back - Cost and throughput estimates, privacy and prompt-injection handling, versioning, and monitoring for drift ### Follow-up Questions - Many records have only a generic name and an address. How do you avoid confident but wrong labels for them? - A new industry is added to the list. How do you relabel the affected businesses without reprocessing everything? - The LLM provider updates its model and the label distribution shifts. How would you detect it and respond? - A business name contains text that reads like an instruction to the model. How do you keep it from changing the output?