Interpreting metrics when autoplay videos reduce time‑spent but increase DAU
Quick Overview
Evaluates interpretation of an autoplay-video experiment where per-session time falls while DAU rises. Strong answers propose behavioral hypotheses, decompose total time, validate segments, and define follow-up experiments and guardrails.
Interpreting metrics when autoplay videos reduce time‑spent but increase DAU
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
Scenario: Enabling autoplay snippets shortens individual sessions yet bumps daily actives. Provide behavioural hypotheses and data tests to explain this paradox and inform product tuning.
Question 1: Auto‑play videos reduce time‑spent yet increase DAU—why? (Hint: quick‑scroll behaviour, re‑engagement of light users)
Quick Answer: Evaluates interpretation of an autoplay-video experiment where per-session time falls while DAU rises. Strong answers propose behavioral hypotheses, decompose total time, validate segments, and define follow-up experiments and guardrails.
You are analyzing an A/B test where short autoplay video previews were enabled in feed. Per-session time spent decreased, but DAU increased.
Explain how both outcomes could be true, validate behavioral hypotheses, decompose metrics, and recommend follow-up experiments.
Constraints & Assumptions
DAU is unique users with at least one qualifying activity in a day.
Session time is active seconds within a session.
Interpret the outcome through a metric tree, not one metric in isolation.
Include quality and retention guardrails.
Clarifying Questions to Ask Guidance
Did total time per user or total time per day change?
Did session count per DAU change?
Which user segments drove the DAU increase?
Did autoplay affect data usage, performance, or user complaints?
Part 1 - Behavioral Hypotheses
What could explain shorter sessions and higher DAU?
What This Part Should Cover Guidance
Include quick-scroll behavior, shorter but more frequent sessions, light-user reactivation, notification or curiosity effects, better task completion, or novelty effects.
Consider negative interpretations such as lower content depth or fatigue.
Part 2 - Analyses and Decomposition
What data tests and metric decomposition would you run?
What This Part Should Cover Guidance
Decompose total time as DAU times sessions per DAU times time per session.
Analyze user-level distributions, new versus returning users, frequency, retention, session count, and content engagement.
Check autoplay exposure, scroll depth, skips, completion, hides, and performance.
Part 3 - Follow-up Experiments
What would you test next and which guardrails matter?
What This Part Should Cover Guidance
Test autoplay thresholds, preview length, mute, ranking, user controls, or segment-specific rollout.
Monitor retention, satisfaction, hides, reports, data usage, battery, latency, and creator outcomes.
Follow-up Questions Guidance
What if DAU lift is concentrated among low-value one-time visits?
How would you distinguish novelty from durable behavior?
What metric would you recommend for the launch decision?