Finding which user behaviors truly cause retention, from correlation to experiments

Quick Overview

A Microsoft product manager analytics question on finding what really drives user retention. Tests framing the outcome and observation window, generating candidate drivers, modeling retention without leakage, separating correlation from causation with experiments, and ranking drivers by their impact.

Finding which user behaviors truly cause retention, from correlation to experiments

Company: Microsoft

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

How would you find out what actually drives user retention for a product? ```hint Separate markers from causes Retained users tend to do more of everything. Think about how you would tell an action that keeps people coming back from one that only marks people who were going to stay anyway. ``` ### Clarifying Questions - Which product, and which retention measure should the analysis explain: week-4 retention of new users, renewal by paying organizations, or something else? - Is the focus on early-life drivers that onboarding can influence, or on why long-time users eventually leave? - What data is available (product event logs, account and billing data, surveys, support tickets), and can we run experiments in the product? - For a product that organizations deploy, should drivers be found at the level of the individual user or of the whole organization? ### What a Strong Answer Covers - A precise outcome definition (retention measure, cohort, horizon) and the time window in which candidate drivers are observed. - Candidate drivers generated from both qualitative sources and behavioral data, across onboarding, core usage, social and quality factors. - Descriptive and multivariate analysis that controls for user type and acquisition source without leaking the outcome. - A path from correlation to causal evidence through experiments or quasi-experiments. - Ranking drivers by how much retention they could add, and turning them into product bets. ### Follow-up Questions - Users who install the mobile app retain twice as well as those who do not. Would you push everyone to install it, and what would you check first? - An experiment that nudged users toward a driver behavior raised the behavior but not retention. What could explain it? - How would the drivers differ between users who chose the product themselves and users whose employer rolled it out?

Overview: A Microsoft product manager analytics question on finding what really drives user retention. Tests framing the outcome and observation window, generating candidate drivers, modeling retention without leakage, separating correlation from causation with experiments, and ranking drivers by their impact.

Microsoft logo
Microsoft
Oct 19, 2026
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

How would you find out what actually drives user retention for a product?

Clarifying Questions Guidance

  • Which product, and which retention measure should the analysis explain: week-4 retention of new users, renewal by paying organizations, or something else?
  • Is the focus on early-life drivers that onboarding can influence, or on why long-time users eventually leave?
  • What data is available (product event logs, account and billing data, surveys, support tickets), and can we run experiments in the product?
  • For a product that organizations deploy, should drivers be found at the level of the individual user or of the whole organization?

What a Strong Answer Covers Guidance

  • A precise outcome definition (retention measure, cohort, horizon) and the time window in which candidate drivers are observed.
  • Candidate drivers generated from both qualitative sources and behavioral data, across onboarding, core usage, social and quality factors.
  • Descriptive and multivariate analysis that controls for user type and acquisition source without leaking the outcome.
  • A path from correlation to causal evidence through experiments or quasi-experiments.
  • Ranking drivers by how much retention they could add, and turning them into product bets.

Follow-up Questions Guidance

  • Users who install the mobile app retain twice as well as those who do not. Would you push everyone to install it, and what would you check first?
  • An experiment that nudged users toward a driver behavior raised the behavior but not retention. What could explain it?
  • How would the drivers differ between users who chose the product themselves and users whose employer rolled it out?