Find and fix metric drops systematically
Overview
This question evaluates a Data Scientist's ability to decompose key metrics, distinguish instrumentation issues from behavioral changes, and design validation experiments within the Analytics & Experimentation domain, testing both conceptual understanding and practical application.
Other
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
0
0
Investigating a YoY Drop in Weekly Average Likes per User
Context
You are a Data Scientist evaluating a social app where the weekly average likes per user (LPU) has declined year over year (YoY). Assume you have access to instrumentation for both numerator and denominator, including events like: like_tap, like_write_success, like_counted, session_start, feed_impression (exposure), and DAU/WAU definitions.
Tasks
- Arithmetic decomposition: Decompose LPU into factors to enumerate mutually exclusive root causes of a YoY decline. Provide a factor tree and highlight cases like stable likes with user count up due to low-engagement new users.
-
Investigation plan: Propose a step-by-step plan to distinguish technical issues from behavioral changes and to pinpoint problematic segments. Include:
- Diagnostics you would run first
- Decision thresholds for escalation
- An action plan for both a "bad" underperforming segment to fix and a "good" overperforming segment to expand
- Validation: Specify how you would validate root causes and proposed fixes using experiments.
Solution
Login required
Show