Design a Recommendation Ranking System
Company: N/A
Role: Machine Learning Engineer
Category: ML System Design
Interview Round: Onsite
You are interviewing for a staff-level machine learning role focused on recommendation systems.
Design an online recommendation ranking system for a consumer app feed. On each request, the system must choose and order items from a large candidate pool for a specific user.
Business context:
- The primary goal is to maximize long-term user engagement.
- Guardrail metrics include latency, content diversity, bad-content rate, and creator fairness.
- The system should support a multi-stage ranking pipeline because the candidate set is too large for a single heavy model.
Please explain:
1. What clarifying questions you would ask first.
2. How you would translate the business goal into ML objectives.
3. The end-to-end architecture, including candidate generation, lightweight ranking, heavy ranking, and reranking.
4. A deep dive on the heavy-ranking stage: features, labels, model choice, training data, and serving constraints.
5. How you would evaluate the system offline and online.
6. What causes offline-online discrepancy in recommendation systems, and how you would detect and reduce it.
Quick Answer: This question evaluates expertise in designing scalable, low-latency machine learning recommendation and ranking systems, covering multi-stage ranking pipelines, model selection, feature and label engineering, evaluation methodologies, and production serving constraints.