Choosing Success Metrics for Google Search on Pandemic Health Queries
Company: Google
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
As a product manager on Google Search, define the metrics you would use to judge how well Search serves people who look up information about COVID-19.
```hint What success looks like here
Picture someone looking up a testing site or the current vaccine rules. Ask whether more time and more scrolling on the results page is good or bad news for that person before you choose any metric.
```
### Clarifying Questions
- Which queries count as COVID-related: symptoms, testing and vaccine locations, case statistics, local rules such as travel or masking requirements, or also general pandemic news?
- Should the metrics judge the special pandemic results Search can show (information panels, statistics, locators), or every result shown for these queries?
- Are these metrics for an ongoing health dashboard, or for deciding whether a specific launch for these queries worked?
### What a Strong Answer Covers
- A goal framed around people reaching accurate, current and authoritative information quickly, kept separate from search volume that the pandemic itself drives
- How COVID intent is detected and split into needs (testing, vaccines, symptoms, rules, statistics) that call for different success signals
- One primary success metric built on satisfied sessions, with supporting metrics for speed, freshness and source authority
- Guardrails for exposure to misleading health content, pandemic results triggered on unrelated queries, and overall Search quality
- How the metrics would be measured and validated, and why engagement-style metrics such as session length mislead for search
### Follow-up Questions
- COVID-related query volume doubles in a week. Is that good, bad or neither for the product, and what would you look at?
- After a launch, your primary metric rises but human raters find more outdated guidance in the top results. Which signal wins, and what do you do?
- Health authorities reverse a recommendation overnight. How should your freshness and quality metrics handle the change?
Overview: A Google Search product manager question about choosing metrics for COVID-related queries. It tests defining what search success means for urgent health information, picking a primary metric with supporting signals, and adding guardrails for misinformation, misfiring results and overall search quality.