Thread-Safe Buffered Batch Writer for Streaming Records into a Database

Records arrive continuously from a stream and must be written to a database. Many threads receive records and hand them to your component concurrently. Writing records one at a time is too slow, but the database client offers a batch API that inserts a list of records in one call. Design and implement a thread-safe component that buffers incoming records and writes them in batches. ```python class Database: def batch_insert(self, records: list) -> None: """Insert all records in one call. May raise on failure.""" class BatchWriter: def __init__(self, db: Database, max_batch_size: int, max_wait_seconds: float): ... def submit(self, record) -> None: ... # called concurrently from many threads def close(self) -> None: ... # write everything still buffered, then stop ``` `max_batch_size` caps the size of one `batch_insert` call, and `max_wait_seconds` bounds how long a record may sit in the buffer when traffic is low. A design that simply sends each record to a thread pool, where each worker writes that single record, is parallel but is not batching; the point is to accumulate records and use the batch API. ```hint Two reasons to flush List the conditions under which a buffered batch must be written, and make sure a quiet period cannot leave records stranded. ``` ```hint Keep the lock short Consider what work must happen while holding the lock that protects the buffer, and what can happen after you release it. ``` ### Constraints and Clarifications - `submit` is called from many threads at once, and `batch_insert` may be slow compared with `submit`. - The database's batch API is the only way to write. ### Clarifying Questions - What delivery guarantee is required if `batch_insert` fails: retry until success, retry a bounded number of times, or report the failure to the caller? - Must records be written in the order they were submitted, globally or per key? - What should `submit` do when the database falls behind and the buffer grows: block, drop, or raise? - Is it safe to write the same record twice after a retry, or does the database need an idempotency key? - Can several batch writes run in parallel, or must the database see one batch at a time? ### What a Strong Answer Covers - A correct thread-safe buffer in which no record is lost or written twice under concurrent `submit` calls. - Flushing on size and on time, and a `close` that drains everything and stops cleanly. - Swapping the buffer out under the lock and performing the slow database call outside it, with a clear choice between a dedicated flusher thread and flushing on the submitting thread. - Backpressure through a bounded buffer, plus retry, ordering, and failure semantics chosen explicitly. - Throughput and latency trade-offs of the batch size and wait time, and the metrics to watch. ### Follow-up Questions 1. How would you let several batches be in flight at once without breaking per-key ordering? 2. If the process crashes, which records can be lost, and how would you prevent that? 3. How would you choose `max_batch_size` and `max_wait_seconds` in production?