Validate Friends' Content Engagement with Experimental Design Metrics
Overview
This question evaluates a data scientist's skills in experimentation, causal inference, metric definition, segmentation, and product analytics by asking for validation of whether friends' content drives higher social engagement using view and reaction logs.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Technical Screen
Analytics & Experimentation
55
0
Evaluating Friends vs Unconnected Content and Designing an Experiment
Context
You own the Info stream (feed). Today, most content is from friends. You are considering introducing unconnected content (from pages/creators the user doesn’t follow). You have impression/view logs and reactions logs (e.g., likes/loves/comments/shares). You want to:
- Validate the hypothesis that friends’ content drives more social engagement than unconnected content.
- If you launch unconnected content in the Info stream for the first time, define success metrics, propose an experimental design, and explain how to interpret the results.
Assumptions (minimal):
- A "view" means a valid impression (meets visibility threshold).
- "Reaction" covers social actions such as likes/loves/comments/shares.
- Posts are labeled with source: friend vs unconnected.
- If unconnected content isn’t yet in the Info stream, comparable exposures exist in other surfaces or prior pilots to analyze.
Tasks
- Using existing views and reactions data, validate whether friends’ content drives more social engagement than unconnected content. Specify metrics, segmentation, and how you would address confounding.
- Propose an A/B experiment to introduce unconnected posts in the Info stream, including primary success metrics, guardrails, statistical considerations, and how you would interpret outcomes.
Solution
Login required
Show