Design Top K ranking system evaluates requirements, scale assumptions, API/data design, architecture, trade-offs, failure modes, and rollout in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
##### Question
Design a system that can continuously identify the Top K elements from a large or streaming dataset. Discuss data structures, scalability considerations, update handling, and how to support high-throughput queries.
Quick Answer: Design Top K ranking system evaluates requirements, scale assumptions, API/data design, architecture, trade-offs, failure modes, and rollout in a realistic interview setting. A strong answer states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
System Design: Real-time Top-K from a Large/Streaming Dataset
Context
You receive a continuous, high-volume stream of events, each referencing an item (e.g., item_id). The system must continuously identify the Top K most frequent items and serve low-latency queries. Assume:
Data volume is large (potentially millions of events per second), item cardinality can be high, and K is small (e.g., 10–1,000).
Queries may request Top K for different time windows (e.g., last 1 minute, 1 hour, 1 day) and potentially by a grouping key (e.g., per region, per tenant).
Results should be near-real-time with bounded staleness.
Requirements
Design a system that:
Ingests a large/streaming dataset and continuously identifies the Top K elements.
Chooses suitable data structures for exact and approximate solutions.