Design How an LLM Detects Questions Beyond Its Training Cutoff and Says So

Design How an LLM Detects Questions Beyond Its Training Cutoff and Says So

Company: Cohere

Role: Machine Learning Engineer

Category: ML System Design

Difficulty: medium

Interview Round: Onsite

Overview: Design how a large language model recognizes questions that depend on information after its training cutoff and tells the user where its knowledge ends. The question tests data construction for calibrated abstention, prompt versus fine-tuning versus retrieval trade-offs, and evaluating both over-refusal and hallucination.