Upstart Senior+ Data Scientist Interview Experience — A Hardcore Stats Screen With the Hiring Manager

In Q4 2024 a recruiter reached out to me on LinkedIn about a staff DS role at their company. After a quick chat we scheduled a one-hour screen with the Hiring Manager (the ML Applications Director).

After a short self-introduction we got into the first part. He asked me to walk him through a project at work where I needed to measure impact but couldn't launch an experiment. I gave him the background, the constraints, the method I used, and the final results. He asked a lot of follow-up questions — why I chose this method to establish causality (we used an ML-based counterfactual), the limitations, the assumptions, potential bias, and why I didn't use a more direct method instead. We then spent a while discussing short-term vs. long-term impact. Even though he's a director, his stats knowledge felt very solid and he was quick on his feet.

Then we moved into the second part, where he gave me two experiment questions.

QA:
Run an experiment with 3 variants, the goal is to maximize the CTP (purchase rate). Here are the numbers:

• A: 150 visits, 43 purchases

• B: 200 visits, 48 purchases

• C: 100 visits, 15 purchases

• Which variant is winning?

• After launching that variant, how would you predict the future CTP rate?

This was a pretty traditional test question. I asked him for some background, told him my assumptions, then calculated the CI and gave him the result — I had to work out the CI by hand on the pad. Then he asked a bunch of follow-up questions. I don't remember the details exactly, but roughly he gave me a bunch of other metrics besides CTP and asked how I'd make an overall recommendation. Finally he asked how I'd predict future CTP. I roughly walked him through my prediction method, he asked about other factors I'd need to consider, and then we dove into my method in more depth.

QB:
You go to a town and visit a school, and you ask 100 kids how many children are in their family. You get the following data:

• 50 say 1
• 20 say 2
• 30 say 3

Now you go to a random house in this town, knock on the door, and ask how many children live there. Assume every family in this town has at least one child. What is your best estimate of the probability that they have exactly one child?

I feel like I answered this one so-so, but I kept validating different assumptions with him and confirming my approach, and told him about the variables I needed to consider. Time was also pretty short at that point, so I gave him a rough result.

The last part was the usual Q&A, where I asked some questions about the team setup.

Compared to interviews at other companies, I felt like this one was pretty old-school, pretty hardcore — it reminded me of some of the more stats-heavy roles from years back. I think I answered so-so overall. But maybe because the director was pretty nice, two days later they told me I passed, and then it was on to the VO. Hope this helps.