Design a News Aggregator Like Google News
Company: Rippling
Role: Software Engineer
Category: System Design
Difficulty: medium
Interview Round: Onsite
Design a news aggregator similar to Google News. The system collects articles from a large number of publishers, recognizes when many articles cover the same event and groups them into one story, and shows users a ranked feed of top stories that they can also browse by topic. The report gives only the name of the problem, so settle scale, freshness and personalization requirements with the interviewer before designing.
```hint From publisher to feed
Think about how your system learns that a publisher has posted a new article, and how often it would need to check each of thousands of sources.
```
```hint Many articles, one story
Ask how you would recognize that dozens of articles from different publishers describe the same event, including copies of the same wire report.
```
### Clarifying Questions
- How many publishers and new articles per day, and how many users and feed requests per second?
- Is the feed the same for everyone in a region and language, or personalized per user?
- How quickly must a new article appear: within minutes for breaking news, or is an hour acceptable?
- Do we show full articles, or headlines and snippets that link to the publisher's site?
- Are search, topic pages and alerts for breaking news in scope?
### What a Strong Answer Covers
- Requirements and scale estimates agreed before the design, with the read-to-write ratio made explicit
- Discovery and crawling: feeds and sitemaps, polling frequency per source, politeness and fetch failures
- Extraction, deduplication of the same article, and clustering of articles into stories
- Ranking of stories by freshness, coverage, source quality and diversity, with personalization as an extension
- Serving a read-heavy load with precomputed feeds and caches, and storage choices for each kind of data
- Bursts of breaking news, failure recovery, freshness monitoring and low-quality sources
### Follow-up Questions
- A major event breaks and hundreds of publishers write about it within minutes. What happens in each part of the system?
- How do you keep one story from absorbing unrelated articles that share a few keywords?
- How would you add personalization without computing every user's feed from scratch on each request?
- A publisher starts posting spam or copies of other publishers' articles. How do you detect and handle it?
Overview: A system design question asking for a news aggregator in the style of Google News that ingests articles from many publishers, groups articles about the same event into stories and serves ranked feeds by topic and region. It tests crawl scheduling, deduplication, clustering, ranking and read-heavy serving at scale.
Read the full Rippling Software Engineer interview experience this question came from