Design a short-video recommendation system

Design a recommendation system for a short-video feed product.

Your answer should cover the full pipeline:

1. Objective and labels: Define what the system should optimize. Discuss possible labels such as watch time, completion rate, likes, shares, follows, hides, reports, and long-term retention. Explain trade-offs between optimizing short-term engagement and long-term user satisfaction.
2. Data and features: Describe the training data you would collect, including impression logs, user actions, user history, creator features, video metadata, embeddings, freshness signals, and real-time context. Explain which features belong in candidate generation versus ranking.
3. Three-stage recommendation architecture: Propose a typical retrieval or candidate generation stage, a ranking stage, and a re-ranking or serving stage. Explain how each stage works and why the separation is useful for latency and scale.
4. Modeling details: Discuss negative sampling, delayed feedback, cold start, exploration versus exploitation, and how to avoid feedback loops or popularity bias.
5. Offline evaluation: Explain how you would evaluate the system offline, including ranking metrics and calibration. Discuss the limitations of offline evaluation when the training data comes from a previously deployed recommender and therefore has logging-policy bias.
6. Online evaluation and guardrails: Propose online experiment metrics and guardrails, including user experience, diversity, fairness, safety, and latency.

Assume this is a large-scale consumer app with millions of users and videos, and that the interviewer wants both ML depth and product judgment.