Design a Service to Compute Each User's Top 10 Songs

Context

You are designing a backend service for a large-scale music streaming platform to compute and serve each user's Top 10 songs from listening events. The system must support both real-time freshness and offline accuracy/reproducibility.

Assume: hundreds of millions of MAUs, peak 1–2M events/sec globally, multi-region deployment, and clients that can buffer and upload events late or out of order.

Requirements

Functional

1. Ingest streaming listening events from clients (mobile/web/TV), handle duplicates, out-of-order and late arrivals, and occasional replays.
2. Define “listen” eligibility (e.g., listened ≥ N seconds, not muted/seek-skipped) and support sessionization (e.g., 30-min gap) for downstream features.
3. Maintain per-user ranking for windows: last 7 days and last 30 days (Top 10 songs). Consider also all-time if helpful.
4. Provide both streaming (low-latency) updates and batch recomputation/backfill for correctness.
5. Serve results via APIs with configurable window, limit, and as-of freshness.

Non-Functional

• Freshness SLA: P95 end-to-end under ~1–2 minutes for streaming updates; daily correctness reconciliation.
• High availability and horizontal scalability.
• Privacy: minimize PII, secure transit/at-rest, support deletion/retention policies and access control.
• Backfill and replay: ability to recompute windows for past dates and correct bad data.

Scope to Cover

• Event ingestion, deduplication, sessionization.
• Counters/windows for last 7/30 days.
• Per-user ranking and tie-breaking.
• Batch vs. streaming trade-offs.
• Storage: hot cache vs. warehouse/lake.
• Freshness SLAs and data quality.
• APIs for retrieval.
• Personalization signals/weights.
• Privacy/PII handling.
• Backfill/replay strategy.