Sales Slump Root-Cause Analysis and the Build-or-Skip Call on a New Feature

Quick Overview

A product execution prompt about a decline in sales: investigate what is driving the drop, then decide whether building a new feature is the right response. It tests metric validation, decomposition and segmentation, opportunity sizing, and designing an experiment with a clear ship-or-stop rule.

Sales Slump Root-Cause Analysis and the Build-or-Skip Call on a New Feature

Company: Shopify

Role: Product Manager

Category: Analytics & Experimentation

Difficulty: medium

Interview Round: Other

Sales are declining. Explain how you would investigate the decline, and then how you would decide whether building a new feature is the right way to respond. ### Clarifying Questions - Whose sales are falling: total sales across Shopify merchants' stores, one group of merchants, or Shopify's own subscription revenue? - How large is the drop, and did it appear suddenly or slide gradually over months? - Is the decline in the number of orders, the average order value, or both? - Is there already a specific feature someone wants to build, or is the question whether to build anything at all? ### Part 1 — Diagnose the decline Walk through how you would investigate the drop and isolate what is causing it. ```hint Find which factor moved Sales can be split into several measurable factors and segments. Locate the factor and the segment that changed before reasoning about why. ``` #### What This Part Should Cover - Checking that the drop is real and not a data or definition problem - Decomposing sales and segmenting to localize the change - Separating internal causes from external ones such as seasonality ### Part 2 — Decide whether to build With a likely cause in hand, explain how you would decide whether a new feature is the right response, and how you would validate that decision. ```hint Weigh the feature against cheaper fixes A feature is only one possible remedy. Ask whether the cause you found is something a product change can move, and what the cheapest test of that remedy would be. ``` #### What This Part Should Cover - Whether the diagnosed cause is addressable by a product change at all - Sizing the opportunity against cost and alternatives - An experiment with a primary metric, guardrails and a decision rule ### What a Strong Answer Covers - An ordered diagnosis rather than a list of guesses - A clear line from the cause found to the remedy chosen - Quantified sizing and an explicit ship, iterate or stop rule - Awareness of traps such as seasonality, mix shift and novelty effects ### Follow-up Questions - If the decline turns out to come from an external factor, such as seasonality or a competitor's promotion, what would you do instead of building? - Your experiment shows a statistically significant but very small lift. How do you decide whether to ship? - How would you check that the feature's gains are not simply sales shifted from another channel?

Overview: A product execution prompt about a decline in sales: investigate what is driving the drop, then decide whether building a new feature is the right response. It tests metric validation, decomposition and segmentation, opportunity sizing, and designing an experiment with a clear ship-or-stop rule.

Shopify logo
Shopify
Feb 4, 2025
mediumProduct ManagerOtherAnalytics & Experimentation
0
0

Sales are declining. Explain how you would investigate the decline, and then how you would decide whether building a new feature is the right way to respond.

Clarifying Questions Guidance

  • Whose sales are falling: total sales across Shopify merchants' stores, one group of merchants, or Shopify's own subscription revenue?
  • How large is the drop, and did it appear suddenly or slide gradually over months?
  • Is the decline in the number of orders, the average order value, or both?
  • Is there already a specific feature someone wants to build, or is the question whether to build anything at all?

Part 1 — Diagnose the decline

Walk through how you would investigate the drop and isolate what is causing it.

What This Part Should Cover Guidance

  • Checking that the drop is real and not a data or definition problem
  • Decomposing sales and segmenting to localize the change
  • Separating internal causes from external ones such as seasonality

Part 2 — Decide whether to build

With a likely cause in hand, explain how you would decide whether a new feature is the right response, and how you would validate that decision.

What This Part Should Cover Guidance

  • Whether the diagnosed cause is addressable by a product change at all
  • Sizing the opportunity against cost and alternatives
  • An experiment with a primary metric, guardrails and a decision rule

What a Strong Answer Covers Guidance

  • An ordered diagnosis rather than a list of guesses
  • A clear line from the cause found to the remedy chosen
  • Quantified sizing and an explicit ship, iterate or stop rule
  • Awareness of traps such as seasonality, mix shift and novelty effects

Follow-up Questions Guidance

  • If the decline turns out to come from an external factor, such as seasonality or a competitor's promotion, what would you do instead of building?
  • Your experiment shows a statistically significant but very small lift. How do you decide whether to ship?
  • How would you check that the feature's gains are not simply sales shifted from another channel?