This is an inference startup. The basic business is to place different stages of inference on different hardware in a forward-deployed model. For example, a company might have many old accelerator cards that no longer have enough compute for other work and use them for decoding.
The role was Member of Technical Staff in machine learning systems. A recruiter approached me. I was already changing jobs and used this company as interview practice; even if I had received an offer, I would not have joined. I did not really understand the business, and the company felt too early-stage. However, while I was changing jobs, I asked Claude to rank all the companies where I was interviewing. It placed Gimlet Labs second, which surprised me.
Recruiter screen: There were two rounds, one with the external recruiter and one with the company's own recruiter.
Phone screen:
I implemented an LRU cache correctly in one pass.
Onsite:
By this point, I had already received another offer that I liked, so I did not prepare specifically for this interview.
System Design 1:
Design an offline inference system.
The main points were keeping prefill and decode together plus routing. The interviewer dug very deeply into routing. I answered along the lines of Dynamo, but he kept going deeper. Even at stronger inference companies, I had not been questioned at this level of detail.
Coding:
Design and implement an algorithm that buckets CPU-cycle statistics into time windows. The rough setup was two time series whose timestamps were not aligned, and we needed to output aligned time windows. My approach was to treat each adjacent segment as a linear function for interpolation, convert both series to a unified set of timestamps, and find the corresponding values.
Technical Deep Dive:
I discussed a project with one of the founders.
System Design 2:
Design an image-retrieval system in which a user could ask a question and receive the most relevant images in the system. Images could be uploaded offline. I genuinely had not worked in this area. I answered from an old toy retrieval-augmented generation project and my multimodal experience at work. The interviewer seemed reasonably satisfied in the end.
More than a week later, I received a call saying that I had failed because my system design signal was not strong enough. I did not know which round caused it. I was still a little surprised because the interviewers had repeatedly praised both my coding and the resume deep dive. The overall experience was average. The recruiter often disappeared while scheduling interviews. Several cofounders on the team were repeat founders who seemed to have worked together for many years, but I did not feel that my vibe matched theirs very well. The unexplained rejection annoyed me. Maybe they wanted someone who understood both inference and infrastructure, including Kubernetes, extremely well.
Discussion
Loading comments…