Parse a Raw CSV String Into a Structure for Downstream Teams, Handling Corrupted Rows

In a 45-minute phone screen, the interviewer describes the problem out loud instead of pasting a prompt, and you take notes. Another engineering team receives data as a raw CSV payload held in a single string. Write code that turns that raw string into a data structure other engineering teams can consume, and design both the structure and the way they will access it. You are shown only one sample record, as a string. Everything else about the format you have to establish by asking. The interviewer wants the parsing written from scratch: saying that a dataframe library could load it in two lines does not count as an answer, so the tokenizer and the structure must be your own code. An illustrative payload (the column names and values are invented for this practice version): ```text vehicle_id,timestamp,speed_mps,note av-17,2024-03-01T10:00:00Z,12.5,"stopped, then resumed" av-22,2024-03-01T10:00:01Z,8.0, ``` ### Clarifying Questions - Is the first line always a header, and are its column names unique and non-empty? - Which dialect is this: is the delimiter always a comma, can fields be quoted, can a quoted field contain commas, doubled quotes or line breaks, and are line endings LF or CRLF? - Should values stay as strings, or do consumers expect typed columns (numbers, timestamps)? Is there a schema, or should types be inferred? - How will the downstream teams use the result: iterate over rows, pull whole columns, look records up by a key, or filter? - Does the whole payload fit in memory, or could it be large enough that it must be processed as a stream? - Is an empty field (two delimiters in a row) different from a missing field? ### Part 1 — Parse the raw string into a structure other teams can use Implement the parser and the data structure for well-formed input. Show the public interface a downstream engineer would call, and walk through the sample record. ```hint Keep the two jobs apart Splitting characters into records and fields is one job; deciding how records are stored and exposed is another. Mixing them makes quoted commas and line breaks hard to get right. ``` ```hint Design from the caller's side Before choosing lists, dictionaries or columns, write down the two or three calls a consumer on another team will make most often. ``` #### What This Part Should Cover - Tokenizing that respects quoting (delimiters and line breaks inside quotes, doubled quotes) rather than naive splitting - A structure with a small, stable access interface that downstream code can depend on - Handling of the header, blank lines, trailing newline and CRLF endings - Time and memory cost of the parse ### Part 2 — Corrupted rows Follow-up: the header can be trusted, but row payloads may be corrupted, for example a row that is missing fields. Extend your solution so that corrupted rows are handled deliberately instead of crashing the parse or silently producing misaligned data. ```hint A short row is ambiguous When a row has fewer values than the header, ask whether you can actually tell which column is missing. ``` #### Clarifying Questions for this Part - Should a corrupted row be dropped, kept with its gaps marked, or should it fail the whole parse? - Do the consumers need to know which rows were bad, on which line, and why? - Besides short rows, can rows also have extra fields or an unterminated quote? #### What This Part Should Cover - Detecting short and long rows against the trusted header width - A clear policy for corrupted rows, ideally configurable, that never shifts values into the wrong column silently - Reporting problems (line number and reason) so consumers can act on them - Keeping one bad row from damaging the rows around it ### What a Strong Answer Covers - Narrating assumptions and trade-offs aloud from the first minutes, since the prompt is deliberately under-specified and delivered verbally - Clarifying the dialect and the consumers before writing code - A hand-written parser that is correct for quoted fields, not just the sample line - Distinguishing an empty value from a missing value in the output - Tests that cover the sample, quoted delimiters, empty fields, short rows and a blank final line ### Follow-up Questions - The payload grows to many gigabytes. How do you change the parser and the structure so consumers can process it without loading everything? - Consumers want typed columns. Where do type conversion and conversion failures live, and how are they reported? - The header itself turns out to contain a duplicated or blank column name. What do you do? - Several teams now depend on your structure. How do you evolve it (new columns, new types) without breaking them?