In Q4 2024, a recruiter reached out to me on LinkedIn about a staff DS role at their company. After a short conversation, we scheduled a one-hour phone screen with the hiring manager (an ML Applications Director).
After a brief self-introduction, we moved into the first part. He asked me to walk him through a project where I needed to measure impact but couldn't launch an experiment. I described the project's background, constraints, the methods I used, and the final results. He asked a lot of follow-up questions — why I chose this particular method to establish causality (we used an ML-based counterfactual approach), the limitations, assumptions, potential biases, and why I didn't choose a more direct method instead. We then spent a while discussing short-term vs. long-term impact. Even though he's a director, his statistics knowledge felt very solid and his reactions were fast.
Then we moved into the second part, where he asked two experiment questions.
QA:
Run an experiment with 3 variants, the goal is to maximize the CTP (purchase rate). And below are the numbers:
A: 150 visit, 43 purchase
B: 200 visit, 48 purchase
C: 100 visit, 15 purchase
- Which variant is winning?
- After launching that, how would you predict the future CTP rate?
This was a classic testing question. I asked him about the background, told him my assumptions, and calculated the CI before giving him the result — I had to hand-calculate the CI on paper for him.
Then he asked some follow-up questions. I've forgotten some of the details, but roughly, he gave me a bunch of other metrics beyond CTP and asked how I'd make a recommendation considering all of them together. Finally he asked how I would predict future CTP. I roughly walked him through my prediction approach, and he asked about a number of other factors I'd need to consider, then dove deeper into my method.
Then the second question:
QB:
You go to a town and visit a school, and you ask 100 kids how many children are in their family. You obtain the following data:
50 say 1
20 say 2
30 say 3
Now you go to a random house in this town, knock on the door, and ask how many children live there. Assume every family in this town has at least one child. What is your best estimate of the probability that they have exactly one child?
I think I answered this one so-so, but I kept validating different assumptions with him throughout, confirming my line of reasoning and going over the variables I needed to consider with him. Time ran short too, so I only gave him a rough result in the end.
The last part was a traditional Q&A, where I just asked some questions about team setup.
Compared to interviews at other companies, I think this one was still pretty old-school and hardcore — it reminded me of some of the more statistics-heavy roles from years back. I feel like I answered so-so overall. But maybe the director is a nice person, because two days later they told me I passed, and then it was on to the VO. Hope this helps everyone.
Discussion
Loading comments…