How would you define and use retention metrics?

Scenario

You are a Data Scientist supporting a consumer product (app or website). A PM asks you to “dive deep” on user retention and recommends tracking 7-day and 28-day retention.

Task

1. Define retention clearly . Give at least three common retention definitions and explain how they differ:
   • N-day (classic/cohort) retention
   • Rolling retention (a.k.a. unbounded)
   • Return rate / weekly active retention (or another reasonable variant)
2. Explain how you would compute and interpret 7-day vs 28-day retention :
   • What user cohorting would you use (e.g., signup/install week)?
   • What does each metric capture about user behavior?
3. Discuss short-term vs long-term tradeoffs :
   • Give examples of product changes that might increase 7-day retention but harm 28-day retention (and vice versa).
   • Propose a metric hierarchy: primary , diagnostic , and guardrail metrics.
4. Call out at least five pitfalls/edge cases when measuring retention and how you would address them (e.g., right-censoring, seasonality, re-installs, bots, changing definitions, missing events, timezone issues).
5. If the PM wants to run an A/B test to improve retention, outline how you would design and evaluate it (unit of randomization, experiment duration, how to handle delayed effects, and any variance reduction or sequential testing considerations).