InVision Usage Investigation: Root-Causing a 35% Decline

Quick Overview

A product manager analytics question about a 35% decline in usage on InVision, the design prototyping and collaboration platform. It tests defining the metric and baseline, ruling out tracking problems, segmenting the decline, and checking internal and external root causes in a sensible order.

InVision Usage Investigation: Root-Causing a 35% Decline

Company: InVision

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

You are a product manager at InVision, the platform designers use to build and share interactive prototypes and collect feedback on them. Usage has fallen by 35%. Walk through how you would work out why. ```hint Is it real, and where is it? Before you reach for product explanations, ask what would have to be true for the 35% to be a measurement artifact, and which group of users or which product the decline is concentrated in. ``` ### Clarifying Questions - Which measure of usage fell: active users, prototypes created, prototype views by the people designers share with, or comments and feedback? - Over what period did it fall, and did it happen overnight or gradually over weeks? - What is the 35% compared against: the previous week, a trailing average or the same period last year? - Is the decline spread across every InVision product and plan, or concentrated in one, such as prototyping, the whiteboard or the developer handoff tool? ### What a Strong Answer Covers - Pinning down the metric definition, time window and baseline before forming hypotheses - Ruling out tracking and data-pipeline problems with an independent source of truth - Segmenting the decline by user type (designers versus the reviewers they share with), product, platform, plan, account and geography - Internal causes (releases, outages, pricing changes, broken notifications) and external causes (seasonality, competing tools, customer churn) checked in a sensible order, each with the data that would confirm or rule it out - Sizing the confirmed cause and turning it into a fix or a plan ### Follow-up Questions - The decline turns out to be concentrated in teams that recently adopted a competing browser-based design tool. What would you do next? - How would you tell a drop caused by fewer active designers from one caused by fewer stakeholders opening shared prototypes? - What monitoring would you set up so that a decline like this is caught within days instead of weeks?

Overview: A product manager analytics question about a 35% decline in usage on InVision, the design prototyping and collaboration platform. It tests defining the metric and baseline, ruling out tracking problems, segmenting the decline, and checking internal and external root causes in a sensible order.

InVision logo
InVision
Dec 23, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

You are a product manager at InVision, the platform designers use to build and share interactive prototypes and collect feedback on them. Usage has fallen by 35%. Walk through how you would work out why.

Clarifying Questions Guidance

  • Which measure of usage fell: active users, prototypes created, prototype views by the people designers share with, or comments and feedback?
  • Over what period did it fall, and did it happen overnight or gradually over weeks?
  • What is the 35% compared against: the previous week, a trailing average or the same period last year?
  • Is the decline spread across every InVision product and plan, or concentrated in one, such as prototyping, the whiteboard or the developer handoff tool?

What a Strong Answer Covers Guidance

  • Pinning down the metric definition, time window and baseline before forming hypotheses
  • Ruling out tracking and data-pipeline problems with an independent source of truth
  • Segmenting the decline by user type (designers versus the reviewers they share with), product, platform, plan, account and geography
  • Internal causes (releases, outages, pricing changes, broken notifications) and external causes (seasonality, competing tools, customer churn) checked in a sensible order, each with the data that would confirm or rule it out
  • Sizing the confirmed cause and turning it into a fix or a plan

Follow-up Questions Guidance

  • The decline turns out to be concentrated in teams that recently adopted a competing browser-based design tool. What would you do next?
  • How would you tell a drop caused by fewer active designers from one caused by fewer stakeholders opening shared prototypes?
  • What monitoring would you set up so that a decline like this is caught within days instead of weeks?