Design a multithreaded event logger
Company: Databricks
Role: Software Engineer
Category: Software Engineering Fundamentals
Difficulty: medium
Interview Round: Onsite
Design a multithreaded in-memory event logger for a server application.
Requirements:
- Many worker threads running in the process need to log events concurrently.
- Each event contains:
- A timestamp (to the millisecond).
- A log level (INFO, WARN, ERROR).
- A message string.
- The logger should:
- Provide a method `log(Event e)` that can be safely called from multiple threads.
- Store events in memory in the order they were logged (best-effort ordering by timestamp, but explain your choice).
- Periodically flush events to disk as an append-only log file.
- Support log rotation when a file exceeds a given size limit (e.g., 100 MB).
Tasks:
1. Describe the concurrency model and data structures you would use so that `log(Event e)` has low contention and good throughput.
2. Explain how you would ensure that events are not lost in case of a crash (or clarify the guarantees you provide).
3. Describe how the flushing and log rotation mechanism works, including which thread is responsible and how it interacts with the logging threads.
4. Discuss trade-offs between:
- Using a global lock vs lock-free or queue-based approaches.
- Per-thread buffers vs a global queue.
5. Mention how you would test correctness under concurrency (e.g., ordering, missing logs, duplicates).
Quick Answer: This question evaluates a candidate's competency in concurrent programming, synchronization primitives, ordering guarantees, and durable logging mechanisms for multithreaded systems.