Scenario
You are the product analyst for a messaging platform planning to introduce a group-calling capability. You need to understand user needs, benchmark competitors without direct data, and design a robust experiment to measure impact under network effects.
Questions
-
What survey questions would you ask current and potential users to uncover their needs for group calling?
-
How would you benchmark competitor adoption of group calls when you lack direct access to their data?
-
Propose an experiment to measure the impact of launching group calls: define treatment, control, primary and guardrail metrics. Is total call volume an appropriate primary metric—why or why not?
-
Group calling exhibits strong network effects. If engineering resources prevent a cluster-randomized rollout, how else could you control for contamination and measure causal impact?
Hints: Consider feature importance, willingness to pay, self-reported usage, third-party data, simulations, and causal-inference corrections like diff-in-diff or instrumental variables.