Deciding Without the Right Data: Proxies, Triangulation and New Instrumentation

You need data to make a product decision, but the precise data you want is not available to you. How would you proceed? ```hint Start from the decision Before hunting for substitutes, ask how precise the answer really has to be for this particular decision, and which result would actually change your choice. ``` ### Clarifying Questions - Is the data missing entirely, or does it exist but is out of reach, for example because another team or a partner owns it, or because privacy rules restrict it? - What decision depends on it, and how costly would it be to reverse if it turns out to be wrong? - How much time is there before the decision is due: enough to add logging or run a test, or only a few days? - Would you like an example from my past experience, or a general approach? ### What a Strong Answer Covers - Tying the data need to a specific decision and to the threshold that would change it. - Alternative evidence (proxy metrics, adjacent datasets, public or partner data, sampling, qualitative research, structured estimates) and a check on whether each proxy really tracks the missing quantity. - Ways to get the real data later, such as requesting access, adding instrumentation or running a small test, weighed by time, cost and privacy. - Making and communicating a decision under uncertainty: ranges, sensitivity, reversibility, stated assumptions and a point to revisit. - Respect for the privacy, legal or contractual reasons the data may be unavailable. ### Follow-up Questions - Your proxy metric and a customer survey point in opposite directions. Which do you trust, and how do you resolve the conflict? - A senior stakeholder refuses to decide until the exact data exists. How do you move forward? - The data you need belongs to a partner who will not share raw records. What arrangements could still answer your question?