Four rounds.
Round 1 — Problem Solving (30 min): We partnered with a local grocery store to add a smart cart. Users could use the smart cart to find this store's own products and prices, and at the same time see products and prices from other stores on the Instacart app through the same cart. They asked whether this is a good idea, what the hypothesis would be, which metrics to look at, and how to design the experiment.
Round 2 — Stats & Experimental Design (45 min): A dozen or so quick-fire questions, from easy to hard. Easy ones like: how would you explain p-value to a PM, what's the exact definition of p-value, how do you interpret it, and if you get a p-value of 0.03 how should you decide.
Harder ones like: how do you interpret each parameter in a linear regression that has confounding factors, and how do you use L1 and L2 regularized regression models.
Round 4 — Bar Raiser (60 min): First some behavioral questions, then a case: on Sunday afternoons the number of orders shoppers were accepting dropped by two-thirds — why? This case was actually pretty complicated, and it wasn't something you could just run through with a framework the way Emma Ding's videos describe it. You had to actually understand Instacart's product and its users' pain points, lay out assumptions straight from the shopper/merchant/user angle for what could be going on, and only then define your metrics. My suggestion: treat it like a murder-mystery game.
Round 4 — Project Review (45 min): Before the onsite they gave me a project — basically analyze an A/B test on my own — and then in this onsite round I took the slides I'd prepared and presented them.
Discussion
Loading comments…