Durable Event Writer: Concurrent Appends That Survive Crashes

Write pseudocode for a durable event writer. Many threads in one process call `write(event)` at the same time, and each call returns only when its event is durable: once `write` has returned, the event must still be in the file after the process crashes or the machine loses power. The round tests concurrent programming and file I/O at a low level, and pseudocode is expected rather than runnable code. The report names only the topic, so confirm the interface and the guarantees before writing. An assumed interface: ```text log = EventLog.open(path) seq = log.write(event_bytes) # returns after event_bytes is durable; seq orders the events log.close() ``` ```hint What returning promises List the system calls that must have completed before `write` returns, and what can still be lost if power fails right after a plain write call has returned. ``` ```hint Waiting together Making the file durable is far slower than copying bytes into it. Think about what several threads that are waiting for durability at the same moment have in common. ``` ```hint The end of the file after a crash A crash can leave part of an event at the end of the file. Decide how the next open recognizes where the valid data ends. ``` ### Clarifying Questions - Must events survive only a process crash, or also an operating system crash and a power loss? - Must events appear in the file in the order their `write` calls were made, and does each event need a sequence number? - What latency per write can a caller accept, and what throughput is needed? - Is the writer one process, or do several processes append to the same file? - What should happen when the disk is full or a sync fails? ### What a Strong Answer Covers - A precise durability contract: which calls must finish before acknowledging, including syncing the directory when the file is created - A record format with a length and a checksum, so that torn and partial records are detected - Thread-safe appends with no interleaved bytes and a well-defined order - Sharing one sync among concurrent writers without acknowledging any event early, and the latency and throughput trade-off this creates - Recovery on open: scanning, truncating a torn tail and resuming sequence numbers - Error handling: short writes, a failed sync, a full disk and an unbounded queue of waiting writers ### Follow-up Questions - A sync call returns an error. Can you retry it and then acknowledge the waiting writers? - How would you split the log into segment files without weakening the guarantee? - How can readers follow the log while writers append to it? - Two processes must append to the same log. What changes?