Design a Conversation Log Ingestion Pipeline
Company: Cohere
Role: Data Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Overview: This question evaluates a data engineer's competency in designing robust daily ingestion and ETL pipelines for conversation logs, covering schema and warehouse design, JSON parsing and validation, deduplication and idempotency, error handling, and operational diagnostics such as Spark data skew.