Design a web-based AI playground — a prompt-testing console similar to the **OpenAI / ChatGPT Playground**. A signed-in user types a prompt, picks a model and its generation parameters, runs it, and watches the response **stream** back token by token. They can save the current configuration as a reusable **preset**, reload presets later, update them, and share them. Because this is a front-end-leaning role, go deep on the **client** (state management, streaming UI, cancellation) and also cover the **back-end** (APIs, data model, the model gateway). The interviewer expects you to **drive the discussion and proactively enumerate the unhappy paths** — application-server failures, model-service failures, timeouts, partial responses, cancellation, and retries — rather than waiting to be asked for each follow-up. Design **both the front-end and back-end** end to end. ```hint Where to start Anchor on the core run loop first: prompt + params → request → **streamed** response. Get the happy-path data flow crisp — client state → API service → model gateway → provider → tokens streamed back — before layering presets, history, and failure handling on top. ``` ```hint Streaming transport The server pushes token deltas one-way (server→client). What transport carries them — **Server-Sent Events** or a streaming `fetch`/`ReadableStream` vs. WebSockets — and what does the one-way traffic pattern imply about which is the right default? Once chosen, what kinds of events (beyond raw tokens) does the wire protocol need so the client can drive its UI through to a terminal state? ``` ```hint Decouple from the provider Provider keys can't reach the browser, parameter ranges differ per model, and you'll want to add a third provider later without rewriting the front-end. Where in your architecture should all of that provider-specific knowledge live so the rest of the system never has to know which model is behind a run? ``` ```hint Retry safety — the classic trap Retrying a failed run sounds harmless, but is it always? Ask yourself what changes the moment the **first token has already streamed** to the user — and whether that changes the answer to "is it safe to just run this again?" ``` ### Constraints & Assumptions State your assumptions; reasonable defaults for this discussion: - Web app (desktop + tablet), one authenticated user per session; multi-tenant (individual users and teams/orgs). Not anonymous public traffic. - Responses **stream**; perceived quality is dominated by **time-to-first-token (TTFT)**, not total completion time. Some generations run for tens of seconds and emit thousands of tokens, so requests are **long-lived** open connections. - Multiple model families / providers exist; parameter ranges and streaming wire formats differ per model (temperature, max output tokens, top-p, penalties, stop sequences, system prompt, response format). - Provider API keys are server-side secrets and must **never** reach the browser. - A single run costs real money in provider tokens, so per-user/per-org usage accounting and rate limiting are required. Assume order-of-magnitude scale ~10k–100k DAU unless you refine it. - Presets are user-owned and can be private, team-visible, or shareable; output retention is configurable for privacy. ### Clarifying Questions to Ask - Single-user experimentation only, or shared/team presets and collaborative editing? What is the preset visibility/sharing model (private / team / public)? - Do we persist full prompt + output history per run, or metadata only — and what are the retention and privacy requirements for prompt/response data? - Which models/providers must we support at launch, and do their parameter sets and streaming formats differ? - Is there a hard cap on generation length / cost per run, and who pays (per-user quota, org billing, free tier)? - Do we need cancellation mid-stream, and must an in-flight run survive a page reload / reconnect? - What are the availability and TTFT targets, and is there an existing auth/identity system to reuse? ### What a Strong Answer Covers - **Core run loop** with an explicit client state machine (idle → streaming → completed | failed | canceled) and progressive render of token deltas. - **Front-end architecture**: component decomposition (prompt editor, parameter panel, preset sidebar, output panel, run history), a coherent client state model including a **dirty-vs-saved-preset** distinction, and cancellation via `AbortController`. - **Model parameters** surfaced in the UI (model choice, temperature, max output tokens, top-p, penalties, stop sequences, system prompt, response format) and **where** they are validated. - **Presets**: create / load / update / duplicate / delete / share, dirty-state handling, and visibility. - **Back-end API surface & data model**: preset CRUD, run creation + streaming + cancel endpoints, and schemas for users, presets, runs, and usage records (with run-status enums and token/cost accounting). - **Model gateway**: provider abstraction, normalized stream events, normalized error taxonomy, timeouts, token metering, optional failover. - **Failure handling, proactively enumerated**: server crash, provider timeout / rate-limit / overload / policy-rejection, idle-stream timeout, partial responses, cancellation, and **retry safety**, each with concrete client behavior. - **Scale & ops**: long-lived-connection handling, async/non-blocking servers, proxy/LB timeout tuning, no DB transaction held open during a stream, async/batched run writes, rate limiting, and caching the right things (NOT caching generations by default). - **Observability**: TTFT, total latency, stream-completion rate, provider error/timeout/cancel rates, token usage and cost per user/org/model. - **Security & privacy**: server-side parameter validation, provider-key isolation, output sanitization against injection, authz on shared presets, retention policy. - **Stated trade-offs**: SSE vs. WebSocket; full-output vs. metadata-only retention; single vs. split run/stream/cancel endpoints. ### Follow-up Questions - A user reloads the page mid-generation. How do you let them reconnect to (or recover) an in-flight run instead of losing it, and what does the run lifecycle need to support that? - You're adding a second model provider with a different streaming format and different parameter limits. Walk through exactly what changes — and what stays the same — given your gateway design. - How do you enforce per-user/per-org cost and rate limits without adding latency to TTFT on the hot path? - The model output is rendered as markdown/HTML in the browser. What is your defense against prompt-injected content producing XSS, and how does it interact with streaming partial, untrusted markup?