Design a Logging Library with Pluggable Sinks: Event Table, Cloud Storage, Stdout
Company: Snowflake
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Technical Screen
Design a logging library that application code uses to emit log records. Each client application chooses where its logs go: an event table (rows in a database table of log events), cloud object storage, or standard output. The report gives only this one-line prompt, so the expected volume, delivery guarantees and API shape are open: settle them with the interviewer before designing.
```hint Keep the caller fast
Think about what a call to the logger must do on the application's own thread, and what can happen later on another thread, given that a database insert or an object upload is far slower than printing a line.
```
```hint One interface, three behaviors
Ask what each destination handles best, such as single lines, batched inserts or large files, and what a common interface must hide or expose so the core never special-cases one of them.
```
### Clarifying Questions
- Is this an in-process library linked into many services, and which languages must it support?
- Can one client send logs to several destinations at once, and can it change the choice at runtime?
- How many records per second should one process handle, and how quickly must logs appear in the event table or in cloud storage?
- When a destination is slow or down, should the application block, drop logs, or spill them locally? Are some levels never to be dropped?
- Are records structured (key-value fields) or free text, and does the event table have a fixed schema?
- Must records keep their order, and is at-least-once delivery, with possible duplicates, acceptable?
### What a Strong Answer Covers
- Functional and non-functional requirements agreed first: levels, structured fields, per-client choice of destination, low overhead for the caller, bounded memory
- A small public API and configuration, plus one sink interface with implementations for the event table, cloud storage and stdout, extensible to new destinations
- An asynchronous path: bounded buffers, background workers, batching suited to each sink, and flushing on shutdown
- Backpressure and failure handling: what is dropped or blocked when a queue fills, retries, local spill and fallback
- Per-sink details: table schema and batched, idempotent inserts; object layout, file rolling and atomic uploads; line output for stdout
- Thread safety, ordering, the library's own metrics, and how it is tested
### Follow-up Questions
- The database behind the event table is unavailable for several minutes. What happens to the application and to its logs?
- How would you keep sensitive fields such as tokens or email addresses out of every sink?
- A client wants a fourth destination, such as a message queue. What does adding it involve?
- How do you make sure records written just before the process exits or crashes are not lost?
Overview: Design an in-process logging library whose clients choose to send logs to an event table, cloud object storage or standard output. It tests a pluggable sink interface, asynchronous batching, backpressure, failure handling and flushing on shutdown while keeping the cost of each log call low.