Design a distributed system that ingests telemetry from millions of devices and supports both:
-
Real-time
analytics (near-real-time dashboards/alerts)
-
Batch
analytics (historical queries, aggregates, offline jobs)
Cover:
-
Data ingestion (protocols, auth, buffering)
-
Storage choices for hot vs cold data
-
Stream processing vs batch processing
-
Schema/partitioning strategy
-
Scalability, reliability, and cost considerations
-
A minimal set of APIs/queries you want to support