Design a RAG Ranking Pipeline
Company: Microsoft
Role: Software Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Onsite
Design a retrieval-augmented generation pipeline for Microsoft Teams that helps an AI agent answer a user query by finding the most relevant applications or documents.
Assume the platform has access to application metadata, keyword signals, documentation, and conversation context. For each query, the system should retrieve candidate items, rank them, and return the top `K` results. Optionally, an LLM may generate a grounded final response using the retrieved evidence.
Describe the end-to-end design, including:
- offline data ingestion and indexing
- retrieval and ranking stages
- online serving flow
- how to handle latency, freshness, and access control
- model and product evaluation metrics
- monitoring, fallback behavior, and safety considerations
Quick Answer: This question evaluates expertise in retrieval-augmented generation, information retrieval and ranking, indexing and offline data pipelines, model integration, scalability, latency, access control, evaluation metrics, monitoring, and safety within an ML-backed search or assistant context.