Find and fix metric drops systematically

Quick Overview

This question evaluates a Data Scientist's ability to decompose key metrics, distinguish instrumentation issues from behavioral changes, and design validation experiments within the Analytics & Experimentation domain, testing both conceptual understanding and practical application.

Find and fix metric drops systematically

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Onsite

A social app’s weekly average likes per user is down year‑over‑year. 1) Decompose the metric arithmetically to enumerate mutually exclusive root causes (e.g., likes/user down with stable user count vs. stable likes with user count up due to low‑engagement new users). 2) Propose a step‑by‑step investigation plan that distinguishes technical issues from behavior changes and pinpoints problematic segments; include diagnostics you’d run first, decision thresholds for escalation, and an action plan for both a 'bad' underperforming segment and a 'good' overperforming segment to expand. 3) Specify how you’d validate root causes and fixes with experiments.

Quick Answer: This question evaluates a Data Scientist's ability to decompose key metrics, distinguish instrumentation issues from behavioral changes, and design validation experiments within the Analytics & Experimentation domain, testing both conceptual understanding and practical application.

|Home/Analytics & Experimentation
Oct 13, 2025, 9:49 PM
mediumData ScientistOnsiteAnalytics & Experimentation
2
0

Investigating a YoY Drop in Weekly Average Likes per User

Context

You are a Data Scientist evaluating a social app where the weekly average likes per user (LPU) has declined year over year (YoY). Assume you have access to instrumentation for both numerator and denominator, including events like: like_tap, like_write_success, like_counted, session_start, feed_impression (exposure), and DAU/WAU definitions.

Tasks

  1. Arithmetic decomposition: Decompose LPU into factors to enumerate mutually exclusive root causes of a YoY decline. Provide a factor tree and highlight cases like stable likes with user count up due to low-engagement new users.
  2. Investigation plan: Propose a step-by-step plan to distinguish technical issues from behavioral changes and to pinpoint problematic segments. Include:
    • Diagnostics you would run first
    • Decision thresholds for escalation
    • An action plan for both a "bad" underperforming segment to fix and a "good" overperforming segment to expand
  3. Validation: Specify how you would validate root causes and proposed fixes using experiments.
Loading comments...