System Design: Browser-Only Chatbot With Streaming and No Server-Side Conversation Storage
Context
Design an AI chatbot where all user messages and conversation history exist only in the user's browser. No server-side database stores conversations. The bot's responses must stream to the UI. Page refresh starts a new conversation. User credential handling occurs on the client.
Requirements
Describe and justify:
-
Architecture boundaries (client, edge, backend/LLM)
-
End-to-end data flow
-
Streaming implementation (SSE vs. WebSockets) and how to implement it
-
Session management without a server database
-
Security for user credentials and API access keys
-
Rate limiting without a server database
-
Failure modes, timeouts, and retry strategies
-
Trade-offs of the no-database approach and mitigations