Design a User Behavior and Metrics Monitoring Aggregator

Design a system that collects user-behavior events and monitoring metrics, aggregates them, and makes the results available for inspection. Define the data the system accepts and the questions its aggregates should answer before choosing an architecture. The workload, scale, freshness targets, and query patterns are open for clarification. ### Constraints & Assumptions - Distinguish an individual behavior event from a metric sample or a preaggregated metric. - Specify the aggregation operations and dimensions the system needs; do not assume every field is a safe grouping key. - State your requirements for freshness, late data, duplicates, retention, and accuracy. - Treat volume estimates and operational targets as practice assumptions rather than supplied facts. ### Clarifying Questions to Ask - Which behavior events and metrics are collected, and what decisions or investigations will use them? - Are the primary queries counts, rates, percentiles, unique-user estimates, or something else? - How quickly must aggregates appear, and can recent results be corrected as late data arrives? - Which dimensions may be queried, and what data should be retained or excluded for privacy? ### Part 1 — Collect and Transport Data Propose the producer interface, data envelope, ingestion service, and durable transport. Explain validation, producer failures, retries, and overload. #### What This Part Should Cover - Schemas and timestamps appropriate to each signal type. - Buffering and acknowledgement semantics. - A strategy for duplicate delivery and invalid data. - Backpressure and observability of loss or lag. ### Part 2 — Aggregate and Serve Results Explain how data is grouped into useful aggregates, how those aggregates are stored, and how users query them. Address late or out-of-order observations and operational failures. #### What This Part Should Cover - Window and time semantics, plus mergeable aggregation state. - Correct treatment of the chosen metrics, including any nonadditive quantities. - Storage and indexing aligned with expected queries. - Recovery, dimension cardinality, retention, and the limits of any accuracy guarantee. ```hint An average, a distinct-user count, and a percentile cannot all be combined by averaging the numbers produced by each worker. ``` ### What a Strong Answer Covers The design should trace one signal from producer to query and explain what happens when that path retries or stops. It should make the meaning and accuracy of aggregates explicit, use data structures suited to the requested measurements, and show where cost, freshness, and retention trade off. ### Follow-up Questions - How would you recompute a faulty aggregate after fixing its logic? - What would happen if a client started sending a unique value in a metric label on every request? - How would you expose the difference between a true zero and missing telemetry?