Pinterest is considering adding a horizontally scrollable carousel at the top of the Home feed, similar to Instagram Stories. The carousel may surface recently saved Pins, recommended Pins, or topic collections.
As a Data Scientist, explain how you would evaluate this feature from product, analytics, and experimentation perspectives.
Your answer should address:
-
What product goal(s) you would first clarify, and how those goals would change your evaluation framework.
-
What primary, secondary, and guardrail metrics you would define. Be explicit about trade-offs among metrics such as module CTR, overall Home CTR, saves, downstream engagement, session depth, retention, and ad or monetization impact.
-
What events, dimensions, and logging you would instrument to build the analytical system for this feature.
-
How you would design an A/B test or staged rollout, including the randomization unit, exposure definition, novelty effects, power / minimum detectable effect, and possible sources of bias or confounding.
-
Suppose the feature launches and overall Home-page CTR decreases. What are the most plausible explanations, and how would you distinguish among a logging bug, a denominator effect, cannibalization of the existing feed, segment-specific effects, or a true product regression?