Determine Key Metrics for Circle's Success Evaluation
Overview
This question evaluates a data scientist's competency in defining north-star metrics and metric frameworks, designing retrospective analyses with bias and confounding mitigation, and normalizing time-series engagement signals for fair comparison.
Meta
Aug 4, 2025, 10:55 AM
Data Scientist
Onsite
Analytics & Experimentation
70
0
Circle (Group-like) Feature: Metrics and Retrospective Sizing Analysis
Context
You are evaluating a new social feature called Circle (similar to Groups). Assume:
- Circle is the only feature on the platform, so guardrails/cannibalization are out of scope.
- You only have retrospective (observational) data; you cannot run an A/B test yet.
- You’ve also been shown three time-series plots of total comments divided by total posts for three different products with different user bases.
Tasks
- Define the north-star metric and a concise metric framework to measure Circle’s success.
- Using only retrospective data, decide whether Circle should be optimized for small groups (≈5–6 members) or for large groups (tens to hundreds). Outline the analysis design, key variables, and how you will mitigate bias/confounding.
- You are shown three lines (time series) of total comments per total posts for three different products with different user bases. Can these lines be directly compared? Why or why not, and how would you normalize them for a fair comparison?
Solution
Login required
Show