Design a news feed aggregator

Q: Design a news feed aggregator

This question tests a candidate's ability to design large-scale distributed systems for content ingestion, personalization, and real-time feed delivery. It evaluates system design competency across data modeling, feed generation strategies, ranking, caching, and reliability — core skills assessed in senior software engineering interviews.

Q: How do I approach System Design interview questions?

System Design questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master system design interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty System Design question, commonly asked during Take-home Project rounds at Rippling.

Q: What role is this question designed for?

This question is commonly asked for Software Engineer candidates at Rippling during technical interviews.

Question

Design a Large-Scale News Aggregator and Personalized Feed

Design a production-ready news aggregation and personalized feed platform — think a consumer news reader along the lines of Google News or Apple News. The system ingests articles from thousands of publishers, propagates new content to users' feeds in near real time, and serves a personalized, ranked feed plus search and topic pages, all at scale.

Your design should cover the full system, including:

Ingestion : pull/receive articles from thousands of publishers via RSS/Atom, webhooks, and partner APIs; handle publisher rate limits, retries, transient failures, idempotency, deduplication, content normalization, media handling (images/video), and spam/NSFW detection.
Freshness : near-real-time propagation from publisher to user feed, plus backfill for content missed during outages.
Personalized feed : a per-user ranked feed driven by signals such as followed sources, followed topics, recency, and engagement history, with first-class support for experimentation (A/B tests and holdouts) on ranking and UI.
Search & discovery : keyword search and topic/tag pages, with geo and language filters.
Storage & indexing : hot vs. cold storage, the search index, caching layers, and the core data models.
Feed generation strategy : the pull-vs-push and fan-out-on-write-vs-fan-out-on-read decision, with justification for the chosen approach.
Reliability : multi-region availability, the consistency model, and a disaster-recovery plan with RPO/RTO targets.
Capacity & cost : a back-of-the-envelope sizing pass and cost controls.
APIs, security, privacy : external/internal APIs, rate limiting, auth, and GDPR/CCPA compliance (consent, export, deletion).
Operations : monitoring, alerting, tracing, SLOs, and rollback/incident mitigation.
Rollout & testing : a phased rollout plan and an end-to-end testing strategy (load, chaos, experiment guardrails).

Constraints & Assumptions

State your own numbers and confirm them with the interviewer; the architecture matters more than the exact figures. A reasonable starting point:

Scope : a consumer news reader , not a social network. Users follow sources/topics; there are no user-to-user follows, comments, or re-shares. Engagement = view/click/like/share/hide signals.
Scale (illustrative) : tens of thousands of active publishers; on the order of millions of new articles per day with event-driven peaks (~20×); ~10M MAU / ~3M DAU; several feed opens per active user per day.
Freshness SLA : an article should be visible in eligible feeds in under 5 s end-to-end (publish → feed).
Latency targets : feed read p95 well under ~200 ms; search p95 under ~300 ms.
Availability : highly available (target ~99.9–99.95%), multi-region, eventual consistency across regions acceptable.
Out of scope : end-user authentication/session management is assumed solved upstream; the recommendation/ranking model science can be sketched rather than fully specified.

Clarifying Questions to Ask Guidance

Is this a pure aggregator (users follow sources/topics) or are there social-graph features (user-to-user follows, re-shares) that would change the fan-out characteristics?
What is the exact freshness requirement, and does it apply uniformly or only to breaking/trending content?
What scale should I design for — publishers, articles/day, MAU/DAU, and expected peak multipliers?
What are the read-latency and availability targets, and is eventual consistency acceptable for feeds and engagement?
Which markets/languages/regions must we support at launch, and are there data-residency constraints alongside GDPR/CCPA?
How important is personalization quality at launch vs. a simple recency/follows baseline — i.e. how much ML scope is in v1?

What a Strong Answer Covers Guidance

A strong answer treats this as an end-to-end platform and is explicit about trade-offs, not just a component list. Look for:

Workload analysis : separates write vs. read load, does a credible back-of-the-envelope sizing, and draws the right conclusion (write rate is small; the interesting load is media bytes, indexing, and read fan-out; the hot working set is small enough for a hybrid fan-out).
Ingestion robustness : adaptive polling + conditional GETs, signed webhooks with replay protection, per-publisher rate limits, exponential backoff with jitter, circuit breakers, dead-letter queues, and a clear two-stage dedup (exact via idempotency key, near-dup via SimHash/MinHash clustering of wire stories).
Idempotency as a unifying property : a single stable key + upserts that makes retries, backfill, and cross-region replay all safe.
Feed-generation decision : a reasoned pull-vs-push / fan-out-on-write-vs-read comparison, an activity-keyed hybrid (push to active users, pull for cold ones), and an explicit hot-key escape hatch for very high-follower sources — tied back to the freshness and latency SLAs.
Ranking & personalization : candidate generation from multiple sources (follows, trending/geo, content-similarity), sensible features, a scoring function that starts simple and graduates to a learned model, diversity/dedup reranking, and an online/offline feature store kept in parity.
Experimentation : deterministic bucketing, holdouts, guardrail metrics with auto-rollback.
Data model & storage tiering : concrete schemas; a justified split of transactional (SQL) vs. append-heavy (NoSQL/columnar) data; hot/cold tiering with TTL; caching and CDN; and the inverse followers_of index that the fan-out worker reads.
Search : an inverted index with language/geo analyzers, BM25 + recency/quality boosts, optional personalized rerank, and serving only safety-cleared content.
Reliability & DR : regional cells, a stated consistency model (home-region writes, local reads), graceful degradation to a non-personalized trending feed, and concrete RPO/RTO with how they're achieved.
APIs, security, privacy : clean external/internal API surface, layered authn/z (OAuth/OIDC for users, HMAC/API-key for publishers, mTLS internally), rate limiting, and a real GDPR/CCPA deletion-propagation story.
Operations & rollout : SLOs with burn-rate alerting, tracing with correlation IDs, kill switches/feature flags, and a phased rollout with load/chaos/experiment-guardrail testing.

Follow-up Questions Guidance

A breaking story is published by a source with several hundred thousand active followers, and three such bursts overlap at peak. Walk through exactly what happens in your fan-out path and how you keep it from melting a Redis shard.
The same wire story is published verbatim by 50 different outlets within a minute. How does it appear (or not) in a user's feed, and how does your dedup interact with ranking and diversity?
A user issues a GDPR deletion request. Trace the tombstone through every store (engagements, assignments, features, feed entries, search index, analytics lake) and explain how you prove deletion completed.
Your personalization model and feature store both become unhealthy during a traffic spike. What does the user see, and what are the precise fallbacks and kill switches you trigger?
How would you evolve the ranking from a hand-tuned linear score to a learned model without regressing engagement, and how do holdouts let you measure the long-term effect?

Design a news feed aggregator

Quick Overview

Design a news feed aggregator

Design a Large-Scale News Aggregator and Personalized Feed

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Submit Your Answer to Earn 20XP

Design a news feed aggregator

Quick Overview

Design a news feed aggregator

Design a Large-Scale News Aggregator and Personalized Feed

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Submit Your Answer to Earn 20XP