Aggregate and Query Vehicle Obstacle Data

Design a metrics aggregation and query system for road-obstacle data collected by a small fleet of vehicles. The data may include locations of surrounding objects, but its full schema is not specified. Vehicles upload once per hour, and the overall data volume is at petabyte scale. The system should support fast queries, with different performance and data-granularity expectations for recent five-minute data and data from within the past year. Clarify what “recent” and “granularity” mean before choosing a storage and aggregation policy. Explain how those expectations interact with hourly uploads. ### What a Strong Answer Covers - Event time, upload time, indexing freshness, and the limits imposed by hourly delivery. - A defensible storage, partitioning, and indexing plan based on the actual query dimensions. - Separate recent-data and historical-data access paths, with clearly defined aggregate accuracy and detail. - Handling of large uploads, repeated or late data, evolving schemas, and query cost. ### Follow-up Questions - Can hourly uploads satisfy a query for all observations generated in the last five minutes? - What information would be lost if historical records were replaced by aggregates?