Design a Grounded LLM Assistant for Case Preparation
Company: Distyl
Role: Software Engineer
Category: ML System Design
Difficulty: hard
Interview Round: Technical Screen
# Design a Grounded LLM Assistant for Case Preparation
Design an LLM-based assistant that helps a user prepare a draft response from documents related to a legal dispute. The system may summarize documents, identify claims and supporting passages, organize a timeline, and draft text for review. It must not present itself as a lawyer or treat generated text as authoritative legal advice.
Cover notebook-style interaction, ingestion, retrieval, grounding, prompt and tool boundaries, evaluation, privacy, and human review. The assistant can answer questions but must not autonomously file, send, or submit a response.
### Clarifying Questions to Ask
- Which document types and jurisdictions are in scope?
- Who is authorized to upload and view each matter?
- Must every factual statement cite a source passage?
- What retention, deletion, and audit requirements apply?
- Which actions always require a qualified human reviewer?
### What a Strong Answer Covers
- Matter-level access isolation and encrypted document storage
- Parsing with provenance down to page or passage
- Retrieval scoped to the current matter and resistant to prompt injection in documents
- Draft generation that separates sourced facts, user assertions, and uncertainty
- Citations that are mechanically checked against retrieved passages
- Human approval, version history, and no autonomous external action
- Evaluation for faithfulness, omission, citation quality, privacy, and unsafe overclaiming
- Redaction, retention, deletion, incident response, and audit controls
- Clear product warnings without relying on warnings as the only safeguard
### Follow-up Questions
- How would you detect a citation that does not support the sentence?
- What if two uploaded documents contradict each other?
- How would you defend against malicious instructions embedded in evidence?
- Which data should never be used for model training?
Quick Answer: Design a grounded LLM assistant that organizes legal-dispute documents and drafts case-preparation material for human review. Address matter isolation, passage provenance, citations, contradictory evidence, document prompt injection, evaluation, privacy, retention, auditability, and strict limits on advice and external actions.