Convince Product Manager to Launch 'Show Similar Products' Button
Overview
This question evaluates a data scientist's skills in experimental design, causal inference, metric selection, and historical modeling using observational interaction logs, and falls under the Analytics & Experimentation domain for a Data Scientist role.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Technical Screen
Analytics & Experimentation
3
0
Scenario
Instagram is considering adding a "Show similar products" button on product-tagged content to boost shopping-related engagement. The feature has not launched yet.
Task
Convince a Product Manager the feature is worth testing and design an experiment, using only existing interaction data (no feature data yet), to estimate potential impact and make a launch decision.
Questions
- What proxy metric(s) from existing logs would you use to estimate the feature’s likely impact on engagement, and why do they map to the proposed button?
- How would you construct historical baselines and simple models to project impact (upper/lower bounds)?
-
How would you design an experiment to decide whether to launch the button, including:
- Randomization unit and any cluster randomization to mitigate network effects
- Control versus treatment definition and exposure eligibility
- Primary metrics, secondary metrics, and guardrail (health) metrics
- Significance level, power, and duration
- What specific criteria would signal it is safe to roll the feature out broadly?
Solution
Login required
Show