Diagnose Decline in User Engagement and Experience Quality

Q: Diagnose Decline in User Engagement and Experience Quality

This question evaluates a data scientist's competency in defining product metrics, diagnosing declines in user engagement through retention, cohort and segmentation analysis, designing experiments or analyses, and applying causal-inference reasoning with observational data.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Product Metrics Deep-Dive and Causal Inference (TikTok)

Context

You are a data scientist working on TikTok’s core product. Over several weeks, daily active users (DAU) have been declining. Separately, a manager wants to understand how users’ network speed affects TikTok usage, but only observational data are available.

Tasks

a) DAU has been steadily declining. Which metrics would you define, and what experiment(s) or analysis would you design to diagnose the issue?

b) To quantify the causal impact of users’ network speed on TikTok usage with observational data, describe a suitable causal-inference approach and how you would explain its validity to a non-technical stakeholder.

Hints: Consider retention, cohorts, segmentation, and methods like instrumental variables (IV) or propensity weighting. Communicate assumptions clearly.

Diagnose Decline in User Engagement and Experience Quality

Product Metrics Deep-Dive and Causal Inference (TikTok)

Context

Tasks

Solution

Comments (0)