Chat System Design and Message Delivery

What's being tested

Interviewers are checking practical mastery of designing a real-time, durable messaging pipeline that balances ordering, durability, and multi-device delivery under partial failures. Expect to demonstrate distributed-systems primitives (persistence, replication, partitioning), client sync protocols, and operational tradeoffs (latency vs durability, ordering vs throughput). Anthropic cares because chat is a microcosm of reliable, user-facing backend services requiring strong correctness, scalability, and clear tradeoff communication.

Core knowledge

• Message persistence: store canonical messages in a durable datastore (e.g., Postgres, Cassandra) with immutable IDs and a monotonic sort key; choose row-store for small scale, wide-column for high write fan-out and TTL requirements.

• Delivery vs storage separation: decouple write path (persist) from fan-out/delivery via a message queue like Kafka or SQS to provide backpressure, retries, and replayable offsets.

• Ordering models: per-conversation causal/total ordering vs per-sender ordering; implement per-conversation sequence numbers or Lamport clocks for causal ordering; enforce ordering within a partition (e.g., one Kafka partition per conversation).

• Idempotency: use client-supplied idempotency keys and server dedup index to guarantee at-most-once semantics for sends; follow Stripe-style idempotency patterns for retries.

• Multi-device delivery: maintain per-device delivery state and offsets; send messages via persistent WebSocket/gRPC streams for online devices and use push notifications for offline wake-up, with server-side replay on reconnect.

• Sync & reconciliation: store per-recipient read/recv receipts and last-seen offsets; on reconnect, client sends last-applied sequence number and server replies with messages > offset plus any membership changes.

• Failure & partial-write handling: prefer a write-ahead pattern: persist, emit to queue, acknowledge to client only after durable persist; use CDC to populate downstream indexes and delivery systems to avoid tight coupling.

• Scalability & partitioning: partition by conversation ID (hot-conversation mitigation by sharding sub-IDs or hashing with sticky routing); model throughput: if avg msg size S bytes and traffic T msgs/sec, bandwidth ≈ TS, and storage growth ≈ TS*retention.

• Consistency vs availability tradeoffs: for global low-latency, accept eventual delivery and reconcile via vector timestamps; for strict ordering across regions, prefer synchronous replication (higher p99s).

• Receipts & read-state: store receipts as compact metadata (per-user highest-seq or per-device set) to avoid per-message writes; for optional per-message receipts, amortize writes with batched updates to indexes.

Tip: keep the canonical message store authoritative and use change-data-capture to feed delivery pipelines and search/index services.

Worked example — Design a One-to-One Chat System

Frame quickly: ask about expected scale (messages/sec, messages/user), retention policy, ordering guarantees (per-conversation total order?), multi-device behavior, and whether receipts are required. Skeleton: (1) persist messages durably with immutable IDs and per-conversation sequence numbers; (2) emit to a queue (Kafka) for fan-out and replay; (3) deliver via per-device streaming connections (WebSocket/gRPC) and push notifications for offline devices; (4) sync on reconnect using last-seen sequence and reconciliation of membership/edits; (5) observability/ops: metrics for p99 delivery latency, consumer lag, and dead-letter queues. Flag a tradeoff: choosing single-partition per conversation simplifies ordering but limits throughput for extremely large groups or extremely hot one-to-one pairs; a sharded sequence or batching protocol can mitigate. Close by saying: if more time, detail schema (message payload, seq, idempotency key), partitioning plan, and sketches of failure scenarios (duplicate, reorder) with recovery protocols.

A second angle — Design a Resilient Chat System

With resilience and group chat emphasis, prioritize fan-out and membership-change handling: use an append-only canonical store plus a fan-out service that builds per-recipient delivery cursors, supporting idempotent replay. For groups, per-conversation ordering becomes harder; pick ordering semantics (per-sender or causal) and implement by assigning logical timestamps and using per-recipient queues to avoid global stalls. Membership changes require careful replay rules: new members should start at join time, removals stop delivery but require audit trails. Also emphasize monitoring consumer lag and automated repairs (rehydrate per-recipient cursors from canonical store when lag exceeds threshold).

Common pitfalls

Pitfall: A tempting design is to directly write to every recipient device synchronously; this blows up latency and availability when any recipient is slow or offline. Instead, persist first and fan-out asynchronously.

Pitfall: Assuming a single global sequence solves ordering; it creates a distributed bottleneck and complex leader election. Prefer per-conversation sequences or partitioned clocks.

Pitfall: Over-indexing per-message receipt writes for every delivery causes write amplification and costs; aggregate receipts into per-user highest-applied offsets or batch updates to reduce pressure.

Connections

This area naturally connects to stream processing and CDC (change-data-capture), mobile sync and conflict-resolution strategies, and observability for distributed systems (consumer lag, p99 delivery latency). Interviewers may pivot to rate-limiting, encryption/key management, or moderation pipelines.

Further reading

• Martin Kleppmann — Designing Data-Intensive Applications — chapters on replication, partitioning, and logs are directly applicable to chat systems.

• Apache Kafka documentation — Exactly-once semantics — practical patterns for durable emit-and-consume pipelines.