Building a credible national brunch ranking from star ratings and review text
Company: Yelp
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
Suppose you can use every rating and review Yelp holds for businesses in the United States. How would you produce a list of the 10 best places in the country to go for brunch? Walk through how you would decide which businesses count as brunch places, how you would score and rank them, and how you would check that the final list is credible.
```hint Watch the small samples
A place with three perfect reviews and a place with thousands of strong reviews are not equally certain. Also ask whether a business's overall rating says anything about its brunch in particular.
```
### Clarifying Questions
- What is the list for: a published feature, a ranking inside search results, or an internal analysis? That changes how much each pick must be explainable and defensible.
- Should the 10 come from anywhere in the country even if a few large cities dominate, or should the list spread across regions?
- Do chains with many locations count, and if so, is each location ranked separately?
- Must a place be open today and currently serve brunch, and how recent must its reviews be to count?
### What a Strong Answer Covers
- A precise way to identify brunch places from the available data (categories, review text, hours) and to measure brunch quality rather than overall quality.
- A score that handles uneven review counts, such as pulling low-volume ratings toward a baseline, with its parameters explained.
- Treatment of review recency, suspicious or low-quality reviews, closed businesses and the bias toward large cities.
- A validation plan: sanity checks, stability of the ranking under parameter changes, and human review before publishing.
- The trade-off between a purely data-driven list and editorial judgment.
### Follow-up Questions
- A new place opened recently and already has 30 glowing reviews. Should it be able to make the list, and how does your method decide?
- A business on your published list turns out to have bought fake reviews. What in your pipeline should have caught it?
- How would you adapt the approach to produce a top 10 for every major city at once?
Overview: An analytical product question that asks how to rank the ten best brunch spots in the United States using complete Yelp ratings and review data. It tests defining the candidate set, adjusting scores for uneven review counts, handling fake or stale reviews, and validating a ranking before publishing it.