Design RAG Retrieval for Data Assets
Company: Databricks
Role: Software Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Technical Screen
Design the retrieval component for an internal AI coding assistant. When a user asks a data-related question, the assistant should identify the most relevant data assets, such as database tables and analysis notebooks, so it can answer accurately.
Cover the following areas:
- What assets and metadata should be indexed?
- How would you build the retrieval and reranking pipeline?
- How would you handle noisy, stale, duplicated, or low-quality notebooks and tables?
- How would the system decide when it has enough evidence to answer versus when it should ask a clarification question?
- What offline and online metrics would you use to evaluate the system?
Quick Answer: This question evaluates competency in designing retrieval components for retrieval-augmented generation, covering information retrieval, metadata modeling, relevance reranking, data quality management, ambiguity detection, and system evaluation.