Metric investigation: why 23andMe's conversion rate fell by a third

Quick Overview

A 23andMe product manager interview question about diagnosing a 33% drop in conversion. It tests how a candidate validates the metric, splits the funnel and traffic mix, separates internal from external causes in a consumer genetic-testing business, and plans how to confirm and fix the cause.

Metric investigation: why 23andMe's conversion rate fell by a third

Company: 23andMe

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

23andMe's conversion has fallen by 33%. Explain how you would find out why, and what you would do once you knew. ```hint Locate the drop before explaining it Settle which conversion fell, over which window, and whether the change is real. Then cut it along the dimensions where a single cause would show up as a concentrated drop. ``` ### Clarifying Questions - Which conversion is this: website visitors buying a kit, buyers registering and returning their kit, or customers upgrading to the paid membership? - Is the 33% a relative fall (for example from 3% to 2%) or a drop in percentage points, and what period is it compared with? - Did the drop happen suddenly on one day, or build up over weeks? - Did visits or orders change too, or only the rate? ### What a Strong Answer Covers - Defining the metric precisely and confirming the drop is real, not a tracking or definition change - A structured split: numerator versus denominator, traffic source, funnel step, device, geography, product and new versus returning visitors - Internal causes such as releases, pricing, checkout, experiments and fulfillment checked before external ones such as seasonality, competitors and news - Hypotheses specific to a consumer genetic-testing business, such as privacy trust, gift-driven seasonality and promotional calendars - A plan to confirm the cause, a fix or mitigation, and the metrics that show recovery ### Follow-up Questions - The whole drop comes from one paid channel and revenue is flat. What do you do next, and is it still a problem? - If the cause were lost trust after privacy news, what product changes would you test to win it back? - How would you tell a post-holiday dip from a lasting decline?

Overview: A 23andMe product manager interview question about diagnosing a 33% drop in conversion. It tests how a candidate validates the metric, splits the funnel and traffic mix, separates internal from external causes in a consumer genetic-testing business, and plans how to confirm and fix the cause.

23andMe logo
23andMe
Nov 30, 2024
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

23andMe's conversion has fallen by 33%. Explain how you would find out why, and what you would do once you knew.

Clarifying Questions Guidance

  • Which conversion is this: website visitors buying a kit, buyers registering and returning their kit, or customers upgrading to the paid membership?
  • Is the 33% a relative fall (for example from 3% to 2%) or a drop in percentage points, and what period is it compared with?
  • Did the drop happen suddenly on one day, or build up over weeks?
  • Did visits or orders change too, or only the rate?

What a Strong Answer Covers Guidance

  • Defining the metric precisely and confirming the drop is real, not a tracking or definition change
  • A structured split: numerator versus denominator, traffic source, funnel step, device, geography, product and new versus returning visitors
  • Internal causes such as releases, pricing, checkout, experiments and fulfillment checked before external ones such as seasonality, competitors and news
  • Hypotheses specific to a consumer genetic-testing business, such as privacy trust, gift-driven seasonality and promotional calendars
  • A plan to confirm the cause, a fix or mitigation, and the metrics that show recovery

Follow-up Questions Guidance

  • The whole drop comes from one paid channel and revenue is flat. What do you do next, and is it still a problem?
  • If the cause were lost trust after privacy news, what product changes would you test to win it back?
  • How would you tell a post-holiday dip from a lasting decline?