Process Sharded Login Logs
Company: Google
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Onsite
You are given user login event logs generated by many servers across multiple regions. The logs are sharded by server or region, so no single machine has the complete globally ordered stream. Events may arrive out of order, may be delayed, and may contain duplicates due to retries.
Discuss how you would process these distributed logs to produce correct global analytics, such as per-user login history or aggregate login counts over time. Cover data ingestion, ordering, deduplication, partitioning, handling late events, and reliability concerns.
Quick Answer: This question evaluates understanding of distributed systems, stream processing, and reliable data engineering concepts related to ingestion, ordering, deduplication, partitioning, late-event handling, and fault tolerance.