Average or Median? Choosing the Summary Statistic for Google Search Metrics

You are interviewing for a product manager role on Google Search. The interviewer asks: when you summarize the metrics you care about, should you focus on the mean or the median? Take a position and explain why. ```hint Picture the spread Pick two or three Search metrics you would actually track and sketch how their values are spread across users or queries before deciding which summary describes them honestly. ``` ### Clarifying Questions - Is the question about one metric, such as search latency or queries per user, or about Search metrics in general? - Will the number go on a dashboard, become a team goal, or be used to judge an A/B test? - Should values be summarized per query, per session or per user? ### What a Strong Answer Covers - A clear position that still ties the choice to the metric's distribution and to the decision the number supports. - How skew, long tails and outliers such as heavy users, automated traffic and slow requests affect each statistic, shown with a concrete Search example. - What each statistic hides, including tail experiences, totals, and metrics where most users have a value of zero. - How the choice plays out in experiments: sensitivity, variance and a suitable test for each statistic. - How the team should report the metric in practice, and whether one number is ever enough. ### Follow-up Questions - After a launch, median search latency improves but mean latency gets worse. What could explain it, and would you ship? - Queries per user rises on average in an experiment, but the gain comes from a small group of accounts. How do you decide whether the result is real and good? - When would neither the mean nor the median be the right statistic to put on the team's goal?