Design a Streaming Metrics Dashboard: Time-Range Queries and Event-Time Watermarks

Design the backend for a dashboard that shows metrics computed from a continuous stream of events and lets users look at any time range. The round was conversational rather than a structured whiteboard exercise, and two topics came up explicitly: how the system answers range queries over time, and how watermarks work. ```hint Do the work before the query Decide what can be aggregated ahead of time so that a query over a long range never has to touch raw events, and which kinds of metrics can still be combined after they have been aggregated. ``` ```hint Event time is not arrival time An event can reach the system well after it happened. Decide when the numbers for a given minute are final, and what happens when a straggler for that minute shows up afterwards. ``` ### Constraints and Clarifications - Not given: what the dashboard displays, event volume, how fresh the numbers must be, how late events can arrive, retention, and how many people view the dashboard. Ask, or state your assumptions. ### Clarifying Questions - Which metrics does the dashboard show (counts, sums, rates, latency percentiles, distinct users), and broken down by which dimensions? - How fresh must the newest data be, and may the last few minutes show provisional numbers that change later? - Which time ranges and granularities do users ask for: the last hour at one-minute resolution, the last quarter at one-day resolution, arbitrary ranges? - How late can events arrive, and must late events still be counted? - What is the event volume, and how long must raw events and aggregates be kept? ### What a Strong Answer Covers - Settles metrics, freshness, lateness tolerance and query ranges before designing - An end-to-end pipeline: ingestion, stream aggregation by event-time window, storage of pre-aggregated data at several granularities, and a query service - Range queries answered from pre-aggregated buckets, with a clear treatment of which metrics merge exactly (counts, sums) and which need special structures (percentiles, distinct counts) - A correct explanation of watermarks: what they assert, how they are generated, what happens to late events, and the freshness-versus-completeness trade-off - Failure handling and observability: replay, idempotent writes, backfills, and the lag of the pipeline itself ### Follow-up Questions - One input partition stops receiving events. What happens to the watermark and to the dashboard, and how do you prevent it? - Users want the p99 latency over the last 30 days. What do you store so that it can be computed without the raw events? - Yesterday's numbers changed after a batch of late events arrived. How do you limit and communicate such corrections? - One dimension value produces most of the events. How do you keep its aggregation from becoming a bottleneck?