Choose Components for a Data Warehouse

# Choose Components for a Data Warehouse Compare Hive, Spark, Hadoop storage and execution, and SQL Server in a data-warehouse environment. Explain which responsibilities each can own, which combinations are sensible, and how workload, scale, latency, governance, and team skills affect the choice. ### Constraints & Assumptions - The platform serves batch transformation, interactive analysis, and recurring reports. - Data volume and concurrency may grow materially. - The organization needs recovery, lineage, and access control. - Avoid treating every named technology as a mandatory layer. ### Clarifying Questions to Ask - Is the workload primarily on-premises, cloud-hosted, or hybrid? - What data volume, query concurrency, and freshness are required? - Are workloads mostly SQL, general distributed processing, or transactional reporting? - Which operational skills and existing licenses are available? ```hint Compare responsibilities, not brand names Separate storage, metadata/catalog, execution, orchestration, and serving before mapping a technology to each role. ``` ### What a Strong Answer Covers - Accurate boundaries among distributed storage, SQL metadata, compute, and relational serving - Workload-specific choices rather than a universal stack - Operational cost, failure recovery, governance, and skill trade-offs - A migration or coexistence path with measurable acceptance criteria ### Follow-up Questions 1. When is Spark an unnecessary complication for a SQL transformation? 2. What makes a metastore a critical reliability boundary? 3. How would you migrate reporting from a relational warehouse without changing metric definitions?