Determine Demand for WhatsApp Group Video-Calls
Overview
This question evaluates proficiency in opportunity sizing, metric hierarchy development, and experimental design for new product features, including clustered A/B testing and interpretation of lift versus risk.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Technical Screen
Analytics & Experimentation
59
0
Opportunity Sizing and Experimentation Plan for WhatsApp Group Video Calls
Scenario
WhatsApp is considering launching a new group video-call feature (not currently available).
Questions
- Using available call-level data, outline how you would determine whether there is demand for group video calls.
- What additional quantitative and qualitative data would you request, and how would you analyze it?
- After launch, list success and guard-rail metrics. If you could keep only one success metric, which one and why?
-
Before launch, engineering asks for an experiment design. Describe step-by-step how you would run and analyze the test, including:
- Randomization unit
- Clustering to reduce spill-over
- Novelty effects
- Criteria for statistical and business significance
Hint: Frame the analysis as opportunity sizing → metric hierarchy → clustered A/B test design → interpretation of lift vs. risk.
Solution
Login required
Show