Parse Server Health-Check Logs and Handle Duplicate and Malformed Records

Servers run periodic health checks, and the results are written to a log. The log is already structured: every record is a set of key–value fields that has been classified in advance, so there is no free-text parsing, only reading keys and values. Write code that reads the records and reports on each server's health. Once the basic version works, the interviewer asks how to treat records that are duplicated and records that do not follow the predefined structure. Both answers depend on what the report is used for. The record format and the exact report were not reported; the illustration below is an assumption to make the parts concrete. ```text records (one JSON object per line; illustrative format) {"host": "node-a", "ts": 0, "status": "pass"} {"host": "node-a", "ts": 60, "status": "fail"} {"host": "node-a", "ts": 60, "status": "fail"} {"host": "node-b", "ts": 0, "status": "pass"} {"host": "node-a", "ts": 120, "status": "pass"} {"host": "node-a", "ts": 180, "status": "pass"} {"host": "node-c", "status": "pass"} host = server name, ts = seconds since the start of the period, status = pass or fail ``` ### Clarifying Questions - Which keys does every record carry, what are their types, and what unit is the timestamp in? - What should the basic report show for each server: its latest status, its counts of passing and failing checks, how long it has been up? - Are records written in time order, or can a server's records arrive late or out of order? - How large is the log, and does the report run once over a file or continuously as records arrive? ### Part 1 — Read the records and summarize each server Read the records and produce a per-server summary, for example each server's latest health-check status and its numbers of passing and failing checks. ```hint Key by server The fields are already structured, so the work is in choosing a per-server structure you can update one record at a time, and in deciding what "latest" means if records are not in time order. ``` #### What This Part Should Cover - Reading key–value records into a typed form, kept separate from the aggregation - A single pass that maintains per-server state, with its time and memory cost - The latest status determined by timestamp rather than by position in the file ### Part 2 — Duplicated records Some records are duplicated. How do you treat them? Explain how the answer changes with the purpose of the report: an operations view that needs to know whether each server is up now, versus a report for a service level agreement (SLA) that must account for the failures in a period. ```hint Ask what the number is for For each purpose, decide which records can change the answer: does an earlier failure still matter once a later check passes, and does seeing the same failure twice change the result? ``` #### Clarifying Questions for this Part - Is a duplicate an identical record delivered twice, or two records for the same server and time that disagree? - For the SLA report, is availability measured by the share of checks that passed or by the time a server was down? #### What This Part Should Cover - A precise definition of when two records are duplicates - The operations view: which records it needs, and why earlier records can be ignored - The SLA view: keeping the history, removing exact copies so that a failure is not counted twice, and computing availability - A policy for duplicates that disagree ### Part 3 — Records that break the predefined structure Sometimes a record arrives that does not follow the predefined structure. How should the code handle it? ```hint Classify the ways a record can break List the ways a record can deviate (not parseable, a missing key, a wrong type, an unknown status value, an extra key) and decide for each whether it can still be used, must be set aside, or should stop the job. ``` #### What This Part Should Cover - Validation at the point where records enter the program, with a specific reason for each rejection - Keeping rejected records visible (quarantine, counts, alerts) instead of silently dropping them or crashing - What a rejected record means for each report, for example a gap in the data rather than a passing check ### What a Strong Answer Covers - Settles the record format and the purpose of each report before writing code - Clean separation of parsing, validation, deduplication and aggregation, so that each policy can change on its own - Policies for duplicates and bad records that are tied to the use of the data and stated explicitly - Correct counting: no double-counted failures and no silently missing servers - Tests for the edge cases in each part ### Follow-up Questions - A server stops reporting entirely. How does each view treat the missing checks? - Records can arrive several minutes late and out of order. What changes in each view? - The log grows to many gigabytes a day from thousands of servers. How do you produce the same reports continuously? - Most records in one file suddenly fail validation after a deployment. What should the job do?