Design a Multi-Agent Chat Backend
Company: Hebbia
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Onsite
Design the backend for an LLM-powered chat product used by customers in a financial-technology setting. The interviewer will show a simple demo of a chat UI, but your scope is the backend.
The system should support:
- A user sending a chat message and receiving a streamed assistant response.
- Conversation history and context management.
- A multi-agent orchestration layer, where different agents may handle tasks such as planning, retrieval, calculation, compliance checks, and final response generation.
- Tool calls to internal services or external APIs.
- Reliability, observability, and safe handling of sensitive financial data.
Describe the architecture, main services, data model, request flow, scaling strategy, and failure handling.
Overview: This question evaluates a candidate's ability to design scalable, reliable backend architectures that integrate LLMs and multi-agent orchestration, covering distributed systems, streaming responses, conversation context management, tool integrations, observability, and secure handling of sensitive financial data.