Sales Slump Root-Cause Analysis and the Build-or-Skip Call on a New Feature
Quick Overview
A product execution prompt about a decline in sales: investigate what is driving the drop, then decide whether building a new feature is the right response. It tests metric validation, decomposition and segmentation, opportunity sizing, and designing an experiment with a clear ship-or-stop rule.
Sales Slump Root-Cause Analysis and the Build-or-Skip Call on a New Feature
Company: Shopify
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
Sales are declining. Explain how you would investigate the decline, and then how you would decide whether building a new feature is the right way to respond.
### Clarifying Questions
- Whose sales are falling: total sales across Shopify merchants' stores, one group of merchants, or Shopify's own subscription revenue?
- How large is the drop, and did it appear suddenly or slide gradually over months?
- Is the decline in the number of orders, the average order value, or both?
- Is there already a specific feature someone wants to build, or is the question whether to build anything at all?
### Part 1 — Diagnose the decline
Walk through how you would investigate the drop and isolate what is causing it.
```hint Find which factor moved
Sales can be split into several measurable factors and segments. Locate the factor and the segment that changed before reasoning about why.
```
#### What This Part Should Cover
- Checking that the drop is real and not a data or definition problem
- Decomposing sales and segmenting to localize the change
- Separating internal causes from external ones such as seasonality
### Part 2 — Decide whether to build
With a likely cause in hand, explain how you would decide whether a new feature is the right response, and how you would validate that decision.
```hint Weigh the feature against cheaper fixes
A feature is only one possible remedy. Ask whether the cause you found is something a product change can move, and what the cheapest test of that remedy would be.
```
#### What This Part Should Cover
- Whether the diagnosed cause is addressable by a product change at all
- Sizing the opportunity against cost and alternatives
- An experiment with a primary metric, guardrails and a decision rule
### What a Strong Answer Covers
- An ordered diagnosis rather than a list of guesses
- A clear line from the cause found to the remedy chosen
- Quantified sizing and an explicit ship, iterate or stop rule
- Awareness of traps such as seasonality, mix shift and novelty effects
### Follow-up Questions
- If the decline turns out to come from an external factor, such as seasonality or a competitor's promotion, what would you do instead of building?
- Your experiment shows a statistically significant but very small lift. How do you decide whether to ship?
- How would you check that the feature's gains are not simply sales shifted from another channel?
Overview: A product execution prompt about a decline in sales: investigate what is driving the drop, then decide whether building a new feature is the right response. It tests metric validation, decomposition and segmentation, opportunity sizing, and designing an experiment with a clear ship-or-stop rule.
Sales are declining. Explain how you would investigate the decline, and then how you would decide whether building a new feature is the right way to respond.
Clarifying Questions Guidance
Whose sales are falling: total sales across Shopify merchants' stores, one group of merchants, or Shopify's own subscription revenue?
How large is the drop, and did it appear suddenly or slide gradually over months?
Is the decline in the number of orders, the average order value, or both?
Is there already a specific feature someone wants to build, or is the question whether to build anything at all?
Part 1 — Diagnose the decline
Walk through how you would investigate the drop and isolate what is causing it.
What This Part Should Cover Guidance
Checking that the drop is real and not a data or definition problem
Decomposing sales and segmenting to localize the change
Separating internal causes from external ones such as seasonality
Part 2 — Decide whether to build
With a likely cause in hand, explain how you would decide whether a new feature is the right response, and how you would validate that decision.
What This Part Should Cover Guidance
Whether the diagnosed cause is addressable by a product change at all
Sizing the opportunity against cost and alternatives
An experiment with a primary metric, guardrails and a decision rule
What a Strong Answer Covers Guidance
An ordered diagnosis rather than a list of guesses
A clear line from the cause found to the remedy chosen
Quantified sizing and an explicit ship, iterate or stop rule
Awareness of traps such as seasonality, mix shift and novelty effects
Follow-up Questions Guidance
If the decline turns out to come from an external factor, such as seasonality or a competitor's promotion, what would you do instead of building?
Your experiment shows a statistically significant but very small lift. How do you decide whether to ship?
How would you check that the feature's gains are not simply sales shifted from another channel?