I'd seen a lot of phone screen reports for this role and passed it pretty easily, but the onsite was like the reports I'd read about — genuinely hard, and unlike anything I'd seen before.
My background is as a DS who has always worked on modeling, with a little bit of product analytics experience mixed in. I don't have hands-on experience running A/B tests, so on the onsite, the experiment design parts — defining metrics and figuring out how to split treatment/control — I answered really poorly.
Case 1 (DS manager): If we wanted to change the feature in the app that automatically pays with credit, how would we test that? This round felt like the interviewer was good at guiding me along, and it felt fairly positive overall.
Behavioral: standard stuff — how do you collaborate, how do you prioritize.
Case 2 (DS manager): In the user search results, if the #1 restaurant and the sponsored restaurant happen to be the same one, the user ends up seeing two identical listings — is that good or bad? How would you test a change to it? This round didn't flow very smoothly with the interviewer; we got stuck a few times in the middle.
Case 3 (the hiring manager, who manages the earlier DS managers): The questions here were pretty scattered, but all centered on promotions. 1) Restaurants can run a promotion like "$5 off $30" — how would you increase restaurant adoption of it? 2) If we wanted to customize this promotion — say "$3 off $15" or "$10 off $50" — how would you design and test that? 3) Given both the customizable promotion option and the old fixed "$5 off $30" option, how would you test which one performs better? This interviewer was very chill — by this point he had his feet up on the chair and had even started eating. I'm guessing he'd already decided I'd failed and figured there was no point keeping up appearances.
Overall, not having actually run an A/B test made it really hard to hit the mark on every question — hard to think through all the angles — and there were a few points during the interview where I nearly laughed out of sheer nervousness. Also curious what other DS folks think: how do you see modeling DS versus product DS being affected by AI going forward?
Discussion
Loading comments…