A Data-Backed Decision That Proved Wrong: Diagnosis, Recovery and Lesson

Quick Overview

A behavioral product manager question asking for a real decision that was backed by data and still turned out wrong. It tests how a candidate diagnoses why the evidence misled them, owns and corrects the mistake, communicates with stakeholders, and changes how they use data afterward.

A Data-Backed Decision That Proved Wrong: Diagnosis, Recovery and Lesson

Company: Amazon

Role: Product Manager

Category: Behavioral & Leadership

Difficulty: medium

Interview Round: Other

Share an example of a decision you made because the data supported it, which later turned out to be wrong. The question is sometimes put more broadly, as a time when data failed you. That wording also takes in data that fell short before any decision went out, data that was missing altogether, and data that was accurate but did not capture how people would react. ```hint Locate the break Before you tell the story, work out exactly where it failed: the data itself, the way you read it, or something that changed after you decided. The lesson you claim has to match that point. ``` ### Clarifying Questions - Should it be a decision I made and owned, or can it be a team decision where I supplied the analysis? - By "wrong", do you mean the outcome was bad, or that the data or my reading of it was flawed? - Does it need to be a product decision, or can it involve pricing, operations or hiring? ### What a Strong Answer Covers - The decision, its stakes, and what the data showed at the time, in enough detail to judge whether the call was reasonable - How the candidate discovered the mistake and how quickly they acted on it - A precise explanation of why the data misled, rather than "the data was bad" - Ownership of the correction and of the communication to stakeholders, without shifting blame - A lasting change in how the candidate uses data, ideally shown in a later decision ### Follow-up Questions - With only what you knew at the time, was it still a reasonable decision? How do you tell a bad decision from a bad outcome? - What check would have caught the problem earlier, and do you use it now? - When the data and your own judgment disagree today, how do you decide which to follow?

Overview: A behavioral product manager question asking for a real decision that was backed by data and still turned out wrong. It tests how a candidate diagnoses why the evidence misled them, owns and corrects the mistake, communicates with stakeholders, and changes how they use data afterward.

Amazon logo
Amazon
Jan 4, 2024
mediumProduct ManagerOtherBehavioral & Leadership
0
0

Share an example of a decision you made because the data supported it, which later turned out to be wrong.

The question is sometimes put more broadly, as a time when data failed you. That wording also takes in data that fell short before any decision went out, data that was missing altogether, and data that was accurate but did not capture how people would react.

Clarifying Questions Guidance

  • Should it be a decision I made and owned, or can it be a team decision where I supplied the analysis?
  • By "wrong", do you mean the outcome was bad, or that the data or my reading of it was flawed?
  • Does it need to be a product decision, or can it involve pricing, operations or hiring?

What a Strong Answer Covers Guidance

  • The decision, its stakes, and what the data showed at the time, in enough detail to judge whether the call was reasonable
  • How the candidate discovered the mistake and how quickly they acted on it
  • A precise explanation of why the data misled, rather than "the data was bad"
  • Ownership of the correction and of the communication to stakeholders, without shifting blame
  • A lasting change in how the candidate uses data, ideally shown in a later decision

Follow-up Questions Guidance

  • With only what you knew at the time, was it still a reasonable decision? How do you tell a bad decision from a bad outcome?
  • What check would have caught the problem earlier, and do you use it now?
  • When the data and your own judgment disagree today, how do you decide which to follow?