Low-Latency Trading System Architecture

What's being tested

These problems test whether you can design a low-latency, deterministic event-processing system while preserving correctness under concurrency, failures, and bursty market traffic. The interviewer is probing practical command of order books, timestamp ordering, partitioning, idempotency, persistence, and lock/contention tradeoffs—not just whether you can draw services on a diagram. Citadel cares because a small latency or correctness regression can affect execution quality, risk exposure, auditability, and the behavior of every downstream trading component.

Core knowledge

• Latency budgets must be decomposed across feed capture, parsing, validation, book update, strategy decision, risk checks, serialization, and network transmission. Measure median, p99, and p99.9; averages hide queueing and garbage-collection pauses.

• A deterministic event loop processes inputs in a defined order, often with one thread owning a symbol or partition. Single-writer ownership avoids locks and makes replay reproducible; parallelism comes from partitioning independent symbols or venues.

• An order book usually stores bids in descending price order and asks in ascending order, with price-time priority. A price level aggregates quantity, while individual orders require stable sequence identifiers for FIFO matching and cancellation.

• For best bid and offer (BBO) and national best bid and offer (NBBO), maintain per-exchange price levels and derive global extrema: highest bid and lowest offer. Lazy-deletion heaps provide efficient updates, but stale heap entries must be validated against the authoritative map.

• Market-data merging over K timestamp-sorted feeds uses a min-heap containing one head event per feed, requiring $O(N \log K)$ time and $O(K)$ auxiliary space. Equal timestamps need an explicit deterministic tie-break, such as venue ID then sequence number.

• Partitioning by symbol, instrument, or instrument-group keeps related state local and preserves ordering. Avoid splitting one order book across workers unless the coordination cost is justified; cross-partition operations require a defined consistency protocol.

• Idempotency prevents duplicate orders or feed events from producing duplicate effects. Carry a client order ID or venue sequence number, retain a bounded deduplication structure, and distinguish replayed acknowledgments from newly applied commands.

• Persistence for auditability is commonly built from an append-only event log plus periodic snapshots. An event-sourced book can reconstruct state by replaying events, while snapshots bound recovery time; the log must preserve ordering, identifiers, and the exact applied decision.

• Pre-trade risk checks belong on the synchronous order path and must fail closed when required state is unavailable. Keep checks local and bounded where possible; never claim an order was accepted before the risk decision and durable handoff semantics are clear.

• Backpressure must be explicit during feed bursts: bounded queues, drop/coalesce policies for non-authoritative market-data views, and circuit breakers. Never silently drop order or audit events; apply sequence-gap detection and trigger recovery instead.

• Concurrency choices depend on ownership and contention. A mutex is simplest and often adequate; a spin lock can reduce sleep/wakeup overhead only for very short, low-contention critical sections; lock-free structures require correct acquire/release or compare-and-swap reasoning and can suffer from ABA and cache-line contention.

• Memory and CPU behavior often dominate microsecond systems: preallocate objects, avoid allocation on the hot path, use contiguous data where practical, pin critical threads, reduce false sharing, and inspect generated code. Benchmark with realistic bursts rather than trusting asymptotic complexity alone.

Worked example: Design a low-latency trading system

A strong candidate first clarifies the instrument universe, order types, throughput and latency targets, required durability, venue connectivity, failure semantics, and whether the system owns matching or routes orders to external venues. They then state assumptions—for example, single-writer processing per instrument, microsecond-scale in-process latency, at-least-once input delivery, and exact audit replay. The answer can be organized around five pillars: feed ingestion and normalization, partitioned order-book or routing state, synchronous risk and order handling, append-only persistence and recovery, and observability/failure recovery. Market-data handlers would validate sequence numbers, update the relevant partition, publish derived BBO events, and record the applied event without putting network or disk waits directly on the critical path. Order handling would use client IDs for idempotency, perform bounded local risk checks, and return an acknowledgment whose meaning—accepted, persisted, or merely queued—is explicitly defined. One key tradeoff is whether to use a single-threaded event loop for determinism or a lock-free shared structure for parallelism; the former usually wins for one book, while partition-level parallelism scales across instruments. The design should also explain what happens after a process crash: load a snapshot, replay the log, reconcile venue sequence numbers, and prevent duplicate external submissions. If there were more time, I would quantify the latency budget with load tests, define packet-loss recovery, and walk through partial failure during an order acknowledgment.

A second angle

For Order Book With Per-Exchange BBO and Cross-Exchange NBBO Queries, the primary challenge is in-memory data-structure design rather than end-to-end service decomposition. The candidate should define update semantics for add, modify, cancel, and delete events, then maintain an authoritative (exchange, side, price) quantity map alongside heaps or ordered maps for best-price queries. Lazy heap deletion is attractive for fast updates, but every peek must discard entries whose current quantity is zero or no longer matches the heap record. The framing should emphasize stale data, equal-price aggregation, exchange isolation, and whether NBBO reads may observe a concurrently updating book. A useful extension is to state how sequence numbers and snapshots make the structure replayable after a feed gap.

Common pitfalls

Treating low latency as only a network problem. Saying “use faster hardware and more threads” is tempting, but contention, allocations, queue buildup, page faults, and tail latency often dominate. Explain the hot path, assign a budget to each stage, and identify what is measured at p99 rather than only quoting throughput.

Pitfall: Claiming a lock-free queue automatically improves performance. If producers contend on the same cache line, or consumers spin while idle, it can be slower and less predictable than a mutex or single-owner queue.

Leaving correctness semantics vague. “Persist the order” can mean accepted into memory, written to a local log, replicated, or acknowledged by a venue. Define the acknowledgment boundary, duplicate behavior, sequence-gap handling, and recovery procedure so the design has an operational contract.

Pitfall: Saying “use Kafka” or “use a database” without explaining whether it sits on or off the critical path, what ordering it provides, and how its latency and failure behavior fit the budget.

Overlooking deterministic replay. A design that is fast in the healthy case but cannot reconstruct the exact book state is weak for trading infrastructure. Preserve event order, stable tie-breakers, identifiers, configuration versions, and the decisions needed to explain an output after the fact.

Connections

The interviewer may pivot to distributed consensus, network protocols and multicast, failure recovery, or the C++ memory model. They may also ask how to benchmark tail latency, design a sequence-gap replay mechanism, or compare event sourcing with replicated state machines.

Further reading

• Designing Data-Intensive Applications — Martin Kleppmann’s treatment of logs, partitioning, replication, consistency, and stream processing.

• Mechanical Sympathy — Martin Thompson’s practical writing on CPU caches, queues, memory barriers, and low-latency JVM/system design.

• The Art of Multiprocessor Programming — rigorous coverage of locks, non-blocking algorithms, linearizability, and concurrent data structures.