System Design Prompt: End-to-End Movie Recommendation System

You are tasked with designing an end-to-end movie recommendation system for a large-scale consumer platform. Assume a web/mobile product with millions of users and a catalog of tens of thousands of titles. Optimize for both user satisfaction and business impact under typical production constraints (latency, scale, and privacy).

Objectives

Clarify and prioritize measurable objectives, for example:

1. Engagement: CTR, play-start rate, watch time, completion rate
2. Satisfaction: rating after watch, thumbs up/down, long-term retention
3. List quality: diversity/novelty/serendipity, coverage
4. Business constraints: content promotion, age/region eligibility, licensing

Signals and Data

Identify key signals and data sources:

1. Explicit feedback: ratings, likes/dislikes
2. Implicit interactions: impressions, clicks, dwell time, watch-time ratio, replays, add-to-list, search queries
3. Context: device, network, locale, time-of-day, session position
4. Content metadata: genres, cast/crew, synopsis text, release year, maturity rating
5. Embeddings: text/video/image embeddings for content similarity
6. User attributes: new vs returning, inferred preferences

Cold-Start Strategy

Describe strategies for:

1. New users: lightweight onboarding, popular/trending, contextual bandits
2. New items: content-based similarity from metadata/embeddings, controlled exploration

Modeling (First Cut)

Propose a first modeling approach such as matrix factorization/NMF for user–item decomposition. State the objective, training procedure (e.g., ALS), and how it integrates into a two-stage system (candidate retrieval + ranking).

Feature Engineering

Outline features for retrieval and ranking:

• User, item, and interaction features
• Sequence/recency features
• Cross features and constraints (eligibility, business rules)

Training Data Creation

Explain how to construct labeled datasets from logs, including:

• Positive/negative definitions
• Negative sampling
• Time-based splits and leakage prevention
• Debiasing (e.g., position bias)

Offline Evaluation

Specify offline metrics and setup:

• Ranking metrics: MAP@K, NDCG@K, Recall@K, HitRate@K
• Time-based validation and cold-start evaluation

Online A/B Testing

Design an A/B plan:

• Primary/secondary metrics and guardrails
• Power analysis, bucketing, and duration
• Data logging and analysis plan

Freshness and Real-time Updates

Describe how user/item vectors and counters stay fresh:

• Streaming updates, approximate real-time personalization
• Incremental/periodic retraining

Rollout and Monitoring

Outline rollout and monitoring:

• Staging/canary/ramp
• Model/data drift, quality dashboards, alerting
• Fallbacks and SLAs