Root-Causing a 25% Drop in Notion's New Free-Plan Accounts

Quick Overview

A metric-diagnosis interview question about a 25% fall in free sign-ups on Notion. It tests whether a product manager validates the data first, breaks the sign-up funnel down by step and segment, separates internal from external causes, and ranks hypotheses with the checks that would confirm each one.

Root-Causing a 25% Drop in Notion's New Free-Plan Accounts

Company: Notion

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

Free sign-ups on Notion are down 25%. How would you work out what caused the drop? ```hint Find where the drop lives Before brainstorming causes, make sure the number is real, then narrow down which step of the sign-up path and which kind of new user account for the missing quarter. ``` ### Clarifying Questions - What period does the 25% cover, and against what baseline: the previous week, the same period last year, or a forecast? - Did the drop appear overnight or build gradually over several weeks? - Does a free sign-up mean any newly created account, or only one that finishes onboarding and creates a workspace? - Have paid-plan, trial or team sign-ups moved too, or only free ones? ### What a Strong Answer Covers - Validating the metric first: its definition, the tracking behind it and a seasonally fair baseline - Locating the drop by funnel step and by segment, such as acquisition channel, region, platform, sign-in method and user type - Checking internal causes (releases, sign-up flow or plan changes, outages, marketing spend, website changes) before external ones (seasonality, competitors, search engine changes) - A ranked list of hypotheses, each paired with the data that would confirm or rule it out - Judging the impact on activated and paying users, and the next actions and monitoring ### Follow-up Questions - The drop turns out to be almost entirely organic search traffic to the website. What do you check next? - Sign-ups fell 25% but new weekly active users barely moved. Is this still a problem? - How would you tell a competitor's launch apart from the effect of a change Notion shipped the same week?

Overview: A metric-diagnosis interview question about a 25% fall in free sign-ups on Notion. It tests whether a product manager validates the data first, breaks the sign-up funnel down by step and segment, separates internal from external causes, and ranks hypotheses with the checks that would confirm each one.

Notion logo
Notion
Jan 3, 2025
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

Free sign-ups on Notion are down 25%. How would you work out what caused the drop?

Clarifying Questions Guidance

  • What period does the 25% cover, and against what baseline: the previous week, the same period last year, or a forecast?
  • Did the drop appear overnight or build gradually over several weeks?
  • Does a free sign-up mean any newly created account, or only one that finishes onboarding and creates a workspace?
  • Have paid-plan, trial or team sign-ups moved too, or only free ones?

What a Strong Answer Covers Guidance

  • Validating the metric first: its definition, the tracking behind it and a seasonally fair baseline
  • Locating the drop by funnel step and by segment, such as acquisition channel, region, platform, sign-in method and user type
  • Checking internal causes (releases, sign-up flow or plan changes, outages, marketing spend, website changes) before external ones (seasonality, competitors, search engine changes)
  • A ranked list of hypotheses, each paired with the data that would confirm or rule it out
  • Judging the impact on activated and paying users, and the next actions and monitoring

Follow-up Questions Guidance

  • The drop turns out to be almost entirely organic search traffic to the website. What do you check next?
  • Sign-ups fell 25% but new weekly active users barely moved. Is this still a problem?
  • How would you tell a competitor's launch apart from the effect of a change Notion shipped the same week?