Determine User Demand for New Video-Calling Feature
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
##### Scenario
A consumer messaging product is preparing to launch an in-app video-calling feature and wants to understand demand, set evaluation metrics, and manage trade-offs.
##### Question
Using only the provided SQL dataset, how would you determine whether users actually need or want the new video-calling feature? If you were granted access to all internal data sources, what additional analyses would you perform to gauge demand for the feature? After release, how would you evaluate or measure the quality and success of the video-calling feature? If the primary success metric improves while a non-primary (guardrail) metric declines, how would you proceed and communicate the decision?
##### Hints
Consider penetration/activation rates, funnel or cohort analyses, A/B test design, metric hierarchy, guardrails, and follow-up monitoring.
Quick Answer: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Determine User Demand for New Video-Calling Feature states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.