Snapchat Machine Learning Engineer Interview Experience — ML Fundamentals, Transformers, and a Botched Course Schedule

The overall process was a standard technical phone screen: they first asked some ML fundamentals and project-related concepts, then asked about transformers, and finished with an algorithm question. Overall I felt the interviewer cared a lot about how solid your fundamentals are, especially ML concepts and coding correctness.

The first part was ML fundamentals, mainly about overfitting and how to address it under different model settings. The interviewer distinguished between the linear model and DNN settings. My answer roughly covered:

1. For linear models, you can use L1/L2 regularization, reduce feature complexity, use cross-validation to select hyperparameters, add more data, do feature selection, etc.
2. For DNNs, you can use early stopping, dropout, weight decay, batch normalization, data augmentation, appropriate initialization, etc.
3. We also touched a bit on the train/validation loss gap: if training loss keeps decreasing but validation loss goes up, that usually signals the model is starting to overfit, and you can address it with early stopping or stronger regularization.

The second part asked about transformers. Mainly about the attention structure and positional encoding. I explained the Q/K/V structure in self-attention, and that the attention score is roughly QK^T / sqrt(d_k), followed by softmax to get the attention weights, then a weighted sum of V. But I didn't answer this part very smoothly, especially positional encoding, where I got a bit stuck. The interviewer followed up asking why transformers need positional encoding, since vanilla self-attention itself is not sensitive to token order and needs extra positional information injected. That can be sinusoidal positional encoding or learned positional embeddings.

Last was coding. The question was LeetCode Course Schedule. Given numCourses and prerequisites, determine whether you can finish all courses — essentially determining whether there's a cycle in a directed graph.

I used DFS, but I didn't handle the visited set / recursion stack correctly, so it didn't pass in the end. The correct approach should use three states:

0 = unvisited
1 = visiting / currently in the DFS path
2 = visited / confirmed no cycle

During DFS, if you encounter a node in the visiting state, that means there's a cycle, so return false; if you encounter a node in the visited state, you can just skip it. After DFS on a node finishes, mark it as visited. You can also use BFS topological sort and check whether you can process all the courses in the end.

Complexity:
Time: O(V + E), where V is the number of courses and E is the number of prerequisites.
Space: O(V + E), mainly for the graph adjacency list and the visited states.

There were mainly two things that tripped me up in this round: first, I wasn't fluent enough on transformer fundamentals, especially positional encoding; second, on a classic graph problem like Course Schedule, I got the visited / recursion stack logic wrong. My suggestion for anyone interviewing after me: make sure you're solid on both DFS cycle detection and topological sort, and be able to walk through ML fundamentals questions clearly split by linear model / DNN / transformer.