Weighing a 10% Amazon Search Gain Against a 2-Second Slowdown
Company: Amazon
Role: Product Manager
Category: Product / Decision Making
Difficulty: medium
Interview Round: Other
You are a product manager on Amazon Search. Your team recently shipped a new feature, and the data shows it boosts Search by 10%. It also adds 2 seconds to load time. What do you do? Walk through how you would interpret both numbers, what you would look at next, and whether you would keep the feature as it is, change it or pull it back.
```hint Decide what success means first
Before weighing the gain against the delay, ask what the 10% counts and whether that number going up means shoppers are finding and buying what they came for.
```
### Clarifying Questions
- What does the 10% measure: the number of searches, clicks on search results, purchases that start from a search, or a relevance score?
- Which load time grew by 2 seconds (the search results page, or the page the search starts from), and what was it before? Is that the typical delay or the slowest users' delay?
- Do the numbers come from a randomized experiment or a before-and-after comparison, and do they cover the website, the mobile app or both?
- Is the feature live for every shopper, or is it still in a test that can be ramped down?
### What a Strong Answer Covers
- A precise reading of what the 10% and the 2 seconds measure, including whether a higher first number is good news
- A goal for Search and a metric hierarchy (one outcome metric plus guardrails) that the decision is judged against
- A funnel comparison, split by device, connection and query type, that shows where the slowdown costs shoppers and where the feature helps
- A breakdown of where the added time comes from and which engineering options could keep the gain without the delay
- A clear recommendation backed by an explicit decision rule, a rollout plan and attention to effects a short test can miss
### Follow-up Questions
- If searches are up 10% but purchases that start from search are flat, what does that tell you about the feature?
- Engineering says most of the delay can be removed, but only after several months of work: what happens to the feature in the meantime?
- How would you detect harm from slower pages that builds up over months and that a short experiment cannot show?
Overview: An Amazon Search product manager prompt: a newly launched feature shows a 10% gain for Search but adds 2 seconds of load time. It tests whether you can pin down what the gain measures, weigh it against the latency cost, diagnose the slowdown and reach a clear keep, fix or roll-back decision.
