Deciding Under Uncertainty: When Product Evidence Is Good Enough to Act

Quick Overview

A product manager interview question on how to judge when the available evidence is good enough to make a decision instead of collecting more. It tests matching the evidence bar to reversibility and stakes, defining decision criteria in advance, triangulating sources, and acting under remaining uncertainty.

Deciding Under Uncertainty: When Product Evidence Is Good Enough to Act

Company: Meta

Role: Product Manager

Category: Product / Decision Making

Difficulty: medium

Interview Round: Other

You are interviewing for a product manager role and are asked how you recognize the point at which you have enough data to make a decision. Explain how you judge that the evidence you hold is sufficient to act, rather than running another analysis, survey or experiment. ```hint Price both mistakes Waiting has a cost and being wrong has a cost, and neither is the same for every decision. Ask what extra evidence could realistically change your choice. ``` ### Clarifying Questions - Is this about a particular kind of decision (shipping an experiment result, picking the next roadmap item, entering a new market), or about how my bar changes across decision types? - Does "data" include qualitative evidence such as user interviews, support tickets and sales feedback, or only metrics and experiment results? - Would you like a general approach, an example from a past decision, or both? ### What a Strong Answer Covers - Sizing the decision first by reversibility, cost of a wrong call and cost of delay, and setting a different evidence bar for each. - Deciding before collecting what the data must answer and which result would change the decision, tied back to the goal the decision serves. - Concrete sufficiency tests: adequately powered experiments, independent sources pointing the same way, data that represents the affected users, and new research that no longer changes the picture. - Acting under the uncertainty that remains, with stated assumptions, staged rollout, guardrails and a planned review. - A specific example that shows the bar being applied, including what was still unknown at the time. ### Follow-up Questions - Your experiment has not reached its planned sample size and a partner launch is due in a week. Do you ship, and on what basis? - User interviews point one way and the dashboard points the other. How do you decide which to trust? - Tell me about a time you decided too early or waited too long. What did you change in how you work afterward?

Overview: A product manager interview question on how to judge when the available evidence is good enough to make a decision instead of collecting more. It tests matching the evidence bar to reversibility and stakes, defining decision criteria in advance, triangulating sources, and acting under remaining uncertainty.

Meta logo
Meta
Mar 31, 2025
mediumProduct ManagerOtherProduct / Decision Making
0
0

You are interviewing for a product manager role and are asked how you recognize the point at which you have enough data to make a decision. Explain how you judge that the evidence you hold is sufficient to act, rather than running another analysis, survey or experiment.

Clarifying Questions Guidance

  • Is this about a particular kind of decision (shipping an experiment result, picking the next roadmap item, entering a new market), or about how my bar changes across decision types?
  • Does "data" include qualitative evidence such as user interviews, support tickets and sales feedback, or only metrics and experiment results?
  • Would you like a general approach, an example from a past decision, or both?

What a Strong Answer Covers Guidance

  • Sizing the decision first by reversibility, cost of a wrong call and cost of delay, and setting a different evidence bar for each.
  • Deciding before collecting what the data must answer and which result would change the decision, tied back to the goal the decision serves.
  • Concrete sufficiency tests: adequately powered experiments, independent sources pointing the same way, data that represents the affected users, and new research that no longer changes the picture.
  • Acting under the uncertainty that remains, with stated assumptions, staged rollout, guardrails and a planned review.
  • A specific example that shows the bar being applied, including what was still unknown at the time.

Follow-up Questions Guidance

  • Your experiment has not reached its planned sample size and a partner launch is due in a week. Do you ship, and on what basis?
  • User interviews point one way and the dashboard points the other. How do you decide which to trust?
  • Tell me about a time you decided too early or waited too long. What did you change in how you work afterward?