Compute User Trust Scores from Activity Logs, Then Merge Each User's History

Given a log of user activity, compute a trust score for every user in it. The exercise has two coding parts followed by a discussion. First, compute the scores from the log alone. Second, compute them from each user's earlier history together with the new log, which means the history has to be retrieved before any score can be computed. Finally, explain how you would take the solution to production and what changes when the input logs are very large. The log format and the scoring rule were not reported; the illustration below is an assumption to make the parts concrete. Use the interviewer's definitions when you have them. ```text log (one event per line; illustrative format) timestamp, user_id, event, device_id 1, u1, login_success, d1 2, u1, login_failure, d9 3, u2, login_success, d4 4, u1, login_success, d2 5, u2, login_failure, d4 trust score (illustrative rule) start at 100 subtract 10 for each distinct device beyond the first subtract 5 for each failed login clamp the result to the range 0 to 100 ``` ### Clarifying Questions - What does each log line contain, which events exist, and can a line be malformed or duplicated? - What is the scoring rule: which signals lower or raise trust, are they counted over all time or over a window, and is the score bounded? - What does a user's earlier history contain: raw past events or stored aggregates? How is it retrieved, and can retrieval fail? - Should the output include users who appear only in the history and not in the new log? ### Part 1 — Trust score from the log Compute the trust score of every user who appears in the log. ```hint One pass, keyed by user For each user, keep only what the rule needs. A signal defined over distinct values needs a structure in which a repeated value counts once. ``` #### What This Part Should Cover - A single pass that builds per-user state, kept separate from the scoring function - Distinct counting versus plain counting for the different signals - Malformed lines, and the time and memory cost ### Part 2 — Include each user's earlier history Each user may also have history from before this log, stored elsewhere and retrieved per user. Retrieve the history first, then compute each user's score from the history and the new log together. With the illustrative rule, assume the history holds each user's earlier devices and failed-login count, for example `u1: devices {d1, d3}, 2 failed logins`, and that `u2` has no history. ```hint Combine before you score Check, signal by signal, whether a value computed from the history and a value computed from the new log can simply be added together. ``` #### What This Part Should Cover - Retrieving history only for the users in the log, once per user and in batches - Correctly merging history and new events for each signal before scoring - New users, and the difference between "no history" and "history could not be retrieved" ### Part 3 — Production and very large logs How would you run this in production, and what changes when the input logs are too large to process in memory on one machine? ```hint Find the key that splits the work Everything so far keeps per-user state in memory. Ask which key lets the work be divided so that each piece can be processed independently, and where the per-user state lives between runs. ``` #### What This Part Should Cover - Streaming the input, and partitioning the work by user when the state does not fit - Joining with history at scale, and storing per-user aggregates for incremental updates - Reliability: idempotent reprocessing, malformed input, and monitoring of the scores ### What a Strong Answer Covers - Settles the log format, the scoring rule and the shape of the history before coding - Clean separation of parsing, per-user aggregation and scoring, so that the rule can change without touching the rest - Correct handling of signals that are not additive when history and new data are combined - A scaling plan built on partitioning by user, with history lookups batched rather than made once per event - Production concerns specific to a score that affects users: reproducibility, versioned rules, monitoring ### Follow-up Questions - A new signal is added to the rule. How do you recompute every user's score, and how do you compare old and new scores? - Scores must update within seconds of a new event instead of once per batch. What changes? - One user, perhaps an automated client, produces a large share of the events. How do you keep that user from slowing down the job? - How would you explain to a support agent why a particular user's score dropped?