Design ChatGPT homepage with streaming choices

Last updated: Jul 31, 2026

Quick Overview

Design ChatGPT homepage with streaming choices evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/ML System Design/OpenAI

Design ChatGPT homepage with streaming choices

OpenAI

Jul 27, 2025, 12:00 AM

hardSoftware EngineerTechnical ScreenML System Design

7

0

Design ChatGPT homepage with streaming choices

System Design: ChatGPT‑Style Homepage with Streaming

Goal

Design a ChatGPT‑style web homepage end to end. Users should type a prompt and see the model’s response stream token‑by‑token in the browser.

Requirements

Frontend
1. Render a chat UI (messages, input box, streaming cursor, retry/stop, multi‑tab resilience).
2. Stream tokens to the UI with low latency and graceful reconnection.
3. Persist conversations and support pagination/search.
Backend
1. Provide a server endpoint that calls a Chat Completions API with streaming.
2. Authenticate users and protect provider credentials.
3. Rate limit users and enforce token/concurrency budgets.
4. Store conversation state (messages, metadata, token counts).
5. Stream tokens to the browser (SSE or WebSockets) with backpressure, retries, and timeouts.
6. Log, metric, and trace requests end‑to‑end; scale under load with cost controls.

Streaming Transport Comparison

Describe when to use SSE vs WebSockets for streaming tokens, including trade‑offs in:

Latency
Reliability and ordering
Backpressure handling
Reconnection semantics
Browser/proxy support and operational complexity

Integration Details

Show how you would integrate a Chat Completions API, including:

Authentication (user identity and server‑to‑provider secrets)
Rate limiting (per user/IP, concurrency, token budgets)
Conversation state storage (schema, summarization, limits)
Streaming tokenization path (upstream to backend to browser)
Error handling and retries (transient vs permanent)
Observability (logs, metrics, traces)
Scalability and cost considerations

Provide concrete architectural choices, sequence of events, and concise code snippets or pseudocode to illustrate the streaming path for both SSE and WebSockets.

Constraints & Assumptions

Preserve the scope, facts, inputs, and requested outputs from the prompt above.
If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

Clarify users, core use cases, read/write patterns, scale, latency, availability, and data retention.
State explicit assumptions before making sizing or architecture decisions.
Prioritize the functional path first, then address reliability, security, observability, and rollout.

What a Strong Answer Covers Guidance

A scoped requirements summary with concrete non-goals and success metrics.
ML-specific data, model, evaluation, serving, and monitoring choices.
Reasoned trade-offs among simple and scalable designs, including bottlenecks and failure modes.
A validation, monitoring, migration, and launch plan appropriate for the risk level.

Follow-up Questions Guidance

What breaks first at 10x traffic or data volume?
How would you degrade gracefully during dependency failures?
What metrics and alerts would prove the design is healthy after launch?

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More ML System Design•More OpenAI•More Software Engineer•OpenAI Software Engineer•OpenAI ML System Design•Software Engineer ML System Design

Your design canvas — auto-saved