##### Question Design a system that can continuously identify the Top K elements from a large or streaming dataset. Discuss data structures, scalability considerations, update handling, and how to support high-throughput queries.

This question evaluates understanding of real-time stream processing, distributed systems scalability, and frequency-estimation data structures for continuously computing Top-K under high throughput, sliding windows, grouping keys, and bounded staleness.

How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

What difficulty level is this interview question?

This is a hard difficulty System Design question, commonly asked during Onsite rounds at LinkedIn.

What role is this question designed for?

This question is commonly asked for Software Engineer candidates at LinkedIn during technical interviews.

Design Top K ranking system | LinkedIn Interview Question

System Design: Real-time Top-K from a Large/Streaming Dataset

Context

You receive a continuous, high-volume stream of events, each referencing an item (e.g., item_id). The system must continuously identify the Top K most frequent items and serve low-latency queries. Assume:

Data volume is large (potentially millions of events per second), item cardinality can be high, and K is small (e.g., 10–1,000).
Queries may request Top K for different time windows (e.g., last 1 minute, 1 hour, 1 day) and potentially by a grouping key (e.g., per region, per tenant).
Results should be near-real-time with bounded staleness.

Requirements

Design a system that:

Ingests a large/streaming dataset and continuously identifies the Top K elements.
Chooses suitable data structures for exact and approximate solutions.
Scales horizontally across shards/partitions.
Handles updates: insertions, deletions/expirations (e.g., sliding windows), out-of-order/late events.
Supports high-throughput, low-latency queries (read path), including caching/materialization.
Discusses consistency, fault tolerance, and operational considerations.

Provide the design, trade-offs, and key algorithms/data structures. Include complexity and accuracy considerations.

Context

Data volume is large (potentially millions of events per second), item cardinality can be high, and K is small (e.g., 10–1,000).

Queries may request Top K for different time windows (e.g., last 1 minute, 1 hour, 1 day) and potentially by a grouping key (e.g., per region, per tenant).

Results should be near-real-time with bounded staleness.

Requirements

Design a system that:

Ingests a large/streaming dataset and continuously identifies the Top K elements.

Chooses suitable data structures for exact and approximate solutions.

Scales horizontally across shards/partitions.

Handles updates: insertions, deletions/expirations (e.g., sliding windows), out-of-order/late events.

Supports high-throughput, low-latency queries (read path), including caching/materialization.

Discusses consistency, fault tolerance, and operational considerations.

Provide the design, trade-offs, and key algorithms/data structures. Include complexity and accuracy considerations.

Design Top K ranking system

Quick Overview

System Design: Real-time Top-K from a Large/Streaming Dataset

Context

Requirements

Solution

Comments (0)

Design Top K ranking system

Quick Overview

System Design: Real-time Top-K from a Large/Streaming Dataset

Context

Requirements

Solution

Comments (0)