Instagram Video Calling: Goals, Metrics and a Launch Experiment Plan

You are a product manager at Instagram, and the team is about to launch video calling. Explain how you would test the feature before it reaches everyone, and how you would measure whether it is a success. ```hint Count a call from both ends A call only happens when one person places it and another picks up. Decide what each of them has to experience for the call to count as a win, and think about which users need the feature for that to happen at all. ``` ### Clarifying Questions - Is this one-to-one calling only, or group calls as well? - Who can call whom: only mutual followers, anyone the user already messages, or any account? - Where are calls started: from a message thread, from a profile, or directly from a post or story? - Do you want a plan for the launch decision, for tracking health after launch, or both? ### What a Strong Answer Covers - A stated goal for video calling and how it serves Instagram and the people on both ends of a call - One primary success metric, precisely defined, with supporting metrics along the caller's and the receiver's journeys - Guardrails and counter-metrics that would catch harm to receivers, to call quality, or to the rest of Instagram and Meta - A test design with a justified randomization unit for a feature that links two users, a duration, and a decision rule set in advance - How the results would be read, including segments and the patterns that would change the decision ### Follow-up Questions - Calls grow quickly in the test, but total time spent on Instagram does not move. How do you read that, and would you still launch? - How would you tell whether Instagram calls are new conversations or ones that moved over from Messenger or WhatsApp, and should that change the decision? - After launch, reports of unwanted calls rise among younger users. What do you check first, and what do you change?