Personalized News Feed System Design (Push + Pull)

Context

You are designing a large-scale personalized news feed for a consumer application. The feed must support both push (server-initiated delivery/notifications) and pull (client-initiated fetch) consumption patterns, with strong ranking, safety, and operational guarantees.

Scale and Latency Targets

• Users: 100M monthly active users (MAU)
• Publishers: 1M
• Ingestion: 200k writes/sec
• Reads: 2M reads/sec
• Latency: p99 < 200 ms for feed reads

Functional Requirements

1. Social graph and controls:
   • Follow/unfollow
   • Mute and block (user-level and publisher-level)
2. Feed quality:
   • Deduplication (exact and near-duplicate)
   • Diversity (content type/source/topic)
   • Freshness/recency constraints
3. Push channel safeguards:
   • Daily notification caps per user
4. Content lifecycle:
   • Content retractions (publisher-initiated takedown)

API Design Deliverables

Design external APIs with request/response schemas and idempotency for:

1. publish
2. subscribe (follow)/unsubscribe
3. get_feed (pagination + consistent cursors)
4. ack_consume
5. retract
6. feedback logging

For each, provide:

• Endpoint path and method
• Request and response JSON schemas
• Idempotency strategy

Architecture Choices and Justification

• Choose fan-out-on-write vs. fan-out-on-read (or hybrid). Justify with the given scale and SLOs.
• Describe storage layers: hot cache/timeline store vs. cold storage.
• Describe event streaming topology.
• Ranking pipeline: candidate generation, features, filtering, scoring, re-ranking, and online inference.

Reliability, Safety, and Operations

• Abuse/spam controls
• Multi-region replication and read locality
• Backfill and replay strategy
• GDPR deletion and data lifecycle
• Rate limiting and quotas
• SLOs, observability, on-call signals
• Failure modes and graceful degradation (including partial outages)

Experimentation

• How to run online experiments on ranking while ensuring user-level traffic consistency across sessions and pagination
• Safe rollout plans and guardrails