Architect cloud data ingestion patterns
Overview
This question evaluates a candidate's ability to design end-to-end cloud data ingestion and serving architectures, including streaming and batch patterns, CDC/event sourcing/micro-batch trade-offs, partitioning and compaction strategies, idempotency, schema evolution, PII tokenization, cost controls, and incident simulation for chaos testing.
Propose a cloud data ingestion and serving pattern for streaming and batch on your preferred cloud (AWS/Azure/GCP). Choose between CDC, event sourcing, or micro‑batch for upstream systems, justify partitioning/compaction, and show how you ensure idempotency, schema evolution (e.g., optional fields), and PII tokenization. Include cost controls (storage tiering, TTL, file size targets) and an incident you would simulate in chaos testing.