Design a live and historical stock quote API fed by a redundant exchange trade feed

Design the market data API for a retail stock-trading app. Clients call two endpoints: - `GET /quotes/{symbol}/live` returns the real-time quote for one symbol, at about 10 million requests per second. - `GET /quotes/{symbol}/historical?range=...` returns the price history of one symbol over the last day, month or year (`range` is `1d`, `1m` or `1y`), at about 1 million requests per second. The only data source is an exchange last-sale trade feed (Nasdaq Last Sale): - It arrives over two independent, redundant, long-lived TCP connections, A and B. - It covers about 7,000 symbols. - Each connection peaks at about 100,000 trades per second. - The market trades 6.5 hours per day. - Each message has four fields: `symbol` (4 to 5 ASCII characters), `timestamp` (epoch microseconds), `price` (a string in "dollars.cents" form) and `seq_no` (starts at 1 each day and increases monotonically). - After a disconnect, a client can replay the feed starting from any `seq_no` of the current day. Design the system end to end: ingesting the feed, serving live quotes and serving historical ranges. ### Constraints and Clarifications - The two connections are redundant copies of one feed. Assume a trade carries the same `seq_no` on both, and confirm it. - Replay covers only the current day. Anything older must come from data you stored yourself. - As described, messages carry no trade size and no bid or ask prices, so the design cannot offer volume or a spread unless the feed provides more fields. - Response contents, latency targets and chart granularity are not given. Ask, or state your assumptions. - `1m` is read as one month, next to one day and one year. Confirm it. ### Clarifying Questions - What does a live quote contain: only the last trade price and its time, or also the day's open, high, low and change from the previous close? - How fresh must a live quote be, measured from the exchange timestamp to the response? - How many points should each historical range return, and at what interval? - Must clients poll these GET endpoints, or may we also offer a streaming connection? - What should each endpoint return outside trading hours, and for an unknown symbol? - Can a price have more than two decimal places? - Can we open extra sessions to the feed (for example for a standby consumer), or only these two connections? ### Part 1 — Ingesting the feed Design the component that reads connections A and B and produces one clean stream of trades for the rest of the system. Address duplicates between the two connections, missing sequence numbers, disconnects, failure of the ingesting process itself, and how prices are represented. ```hint Two copies of one numbered stream `seq_no` tells you both which messages you have already seen and which ones you have not seen yet. Think about what the other connection, and then replay, can do about a number that is missing. ``` #### What This Part Should Cover - Merging A and B into one stream with no duplicates and no losses - Detecting gaps, recovering them with replay, and resuming after the ingesting process crashes - A durable hand-off to downstream consumers, and an exact representation of prices - Throughput at the peak trade rate ### Part 2 — Live quotes at about 10M requests per second Serve `GET /quotes/{symbol}/live` at about 10 million requests per second while keeping quotes fresh. ```hint Compare reads with writes There are about 7,000 symbols and at most about 100,000 trades per second, against 10 million reads per second. Work out how large the whole live state is, and where it could live. ``` #### What This Part Should Cover - Where the live state lives, and how a request is answered without a database round trip - Propagating updates from ingestion to every serving machine, and the staleness this adds - Ordering guarantees, so that a quote never moves backwards - Server restarts, staleness detection, and polling versus streaming ### Part 3 — Historical ranges at about 1M requests per second Serve the historical endpoint for the `1d`, `1m` and `1y` ranges at about 1 million requests per second. ```hint How many different answers are there? Count the distinct (symbol, range) responses and how often each one actually changes, then compare that with 1 million requests per second. ``` #### What This Part Should Cover - Aggregating trades into price bars, and storage for each granularity - Keeping history beyond the current day, which replay cannot provide - Precomputing and caching responses, and refreshing them as bars close - Response size and egress at this request rate ### What a Strong Answer Covers - Clarifies quote contents, freshness and chart granularity before designing - Correct feed handling: A/B arbitration by sequence number, gap recovery through replay, exact prices, and the daily reset of sequence numbers - A clear separation of the write path (about 100,000 trades per second) from the read paths (about 11 million requests per second) - Back-of-envelope numbers that justify the memory, fan-out, cache and egress choices - Failure handling and observability: feed gaps, failover of the ingesting process, stale data detection and cache stampedes ### Follow-up Questions - Clients want every price change pushed to them instead of polling. What changes, and how do you protect the system from slow clients? - A stock splits overnight. How does the historical endpoint stay correct? - Connection A goes silent for 30 seconds while B keeps flowing, then A comes back far behind. What does the ingestion component do, and what alerts fire? - How would you add a five-year range, or volume if the feed starts carrying trade size?