Detect Security Attacks in Text Logs Using Sliding-Window Rules

You are given a large collection of text-format log lines describing network connections, and a set of rules that define security attacks. Write a program that scans the logs and reports every attack the rules detect. The interviewer supplied the rules. One of them: a single source IP address that connects to more than 10 different destination IP addresses within 10 minutes is an attack. The round also covered making a correct but slow first version fast, and producing test cases. AI coding assistants were allowed in this round. ### Constraints and Clarifications - The exact log format and the full rule list were given in the interview but are not known here. Assume each line contains at least a timestamp, a source IP and a destination IP, for example: ```text 2024-03-01T10:00:03Z src=10.0.0.5 dst=10.0.1.20 port=443 action=ALLOW ``` - "More than 10 different destination IPs" means at least 11 distinct destinations; repeated connections to the same destination count once. ### Clarifying Questions - What is the exact line format, and how should malformed lines be handled? - Are the lines sorted by timestamp, or can they arrive out of order? - Does "within 10 minutes" mean any sliding 10-minute interval or fixed 10-minute buckets, and is an interval of exactly 10 minutes included? - How should a detection be reported: once per source IP, once per window, or on every line that keeps the count above the threshold? - What do the other rules look like? Do they share this shape (a count or a distinct count per key within a time window), or do some need different state? - Does the input fit in memory, or must the program stream it? ### Part 1 — Implement the detector Parse the logs and implement the rule above, reporting each offending source IP together with the window in which it crossed the threshold. Structure the code so the other rules from the list can be added. ```hint State per source For each source IP, decide what you must remember so that, when the next line arrives, you can answer "how many different destinations in the last 10 minutes" without re-reading the file. ``` #### What This Part Should Cover - Robust parsing, with a policy for malformed lines - Correct sliding-window semantics, including repeated destinations and the strict "more than" threshold - The alert format and how duplicate alerts are avoided - A rule abstraction that lets other rules plug into the same scan loop ### Part 2 — Make it fast A first version (in the interview, code generated by an AI assistant) produced correct results but performed poorly on large inputs. Analyze where such a detector wastes time and optimize it. ```hint Count the work per line Estimate how much work the program does for each new line as the window fills up, and ask how much of it repeats work already done for the previous line. ``` #### What This Part Should Cover - The complexity of a naive implementation and where the time goes - Incremental window maintenance with bounded work per line - Memory bounds: streaming input and discarding state for idle sources - Measuring the improvement on a large generated input ### Part 3 — Test cases Produce test cases for your detector. ```hint Aim at the edges of the rule Build inputs that sit exactly at the threshold and exactly at the window boundary, and inputs that would fool a detector that counts the wrong thing. ``` #### What This Part Should Cover - Threshold cases on both sides of the limit - Window-boundary cases and equal timestamps - Repeated destinations, interleaved sources, malformed lines and empty input - Randomized comparison against a simple reference implementation, and a performance test ### What a Strong Answer Covers - Clarifies the format, window semantics and alert policy before coding - A correct, readable implementation with an extensible rule structure - Streaming processing with near-constant work per line and bounded memory - Thorough, targeted tests, including a brute-force reference - A critical review of generated code rather than accepting it as is ### Follow-up Questions - Lines can arrive slightly out of timestamp order. How do you keep the results correct? - The logs are too large for one machine. How would you split the work, and what must the data be partitioned by? - How would you add a second rule that counts total connections per source rather than distinct destinations, without duplicating the window logic? - How would you turn this batch program into a real-time detector over a live stream of events?