Design a Jira+Confluence RAG assistant
Company: Atlassian
Role: Machine Learning Engineer
Category: ML System Design
Difficulty: medium
Interview Round: Onsite
You are asked to design a simple Retrieval-Augmented Generation (RAG) system that answers employee questions using content from two internal products: Jira (issues, comments) and Confluence (pages, spaces).
Design an MVP that supports “cross-product” questions such as:
- “What is the current status of project X and what design doc explains the latest decision?”
- “Summarize the open P0 bugs for service Y and link the relevant runbook page.”
Requirements and constraints:
- Data sources: Jira issues (title/description/comments/metadata) and Confluence pages (title/body/metadata).
- Must respect document-level permissions (a user should only retrieve what they can access).
- Latency target: interactive Q&A (a few seconds).
- Results should cite sources (links to Jira issues / Confluence pages).
- Handle updates: Jira changes frequently; Confluence changes less frequently.
Describe:
1) Ingestion and indexing (chunking, embeddings, metadata)
2) Retrieval strategy (filters, ranking, hybrid search)
3) Prompting / generation and citation handling
4) Permission enforcement approach
5) Evaluation and monitoring (offline + online)
Quick Answer: This question evaluates competencies in designing retrieval-augmented generation systems, including ingestion and indexing, retrieval and ranking strategies, prompt-based generation and citation handling, permission-aware access control, and monitoring for freshness and latency.