Reduce LLM hallucination and handle class imbalance
Company: Cvs
Role: Data Scientist
Category: Machine Learning
Difficulty: easy
Interview Round: Technical Screen
Answer the following applied ML/LLM questions.
## 1) LLM hallucination & token cost control
You are building a chatbot over an internal knowledge base.
1. What are common causes of hallucination in LLM chatbots?
2. How would you reduce hallucinations using a RAG-style approach (retrieval + generation)?
3. How would you control or reduce token costs while maintaining answer quality?
Discuss concrete design choices (e.g., chunking, retrieval quality, prompt construction), evaluation ideas, and failure modes.
---
## 2) Class imbalance + precision vs. recall trade-off
You are building a binary classifier where the positive class is rare.
1. How would you handle class imbalance during training and evaluation?
2. In a scenario where **false positives are more costly than false negatives**, which metric should be prioritized (precision vs. recall), and how would you choose an operating threshold?
Explain your reasoning and mention practical checks/pitfalls (e.g., calibration, dataset shift).
Quick Answer: This question evaluates applied machine learning competencies including LLM safety and hallucination mitigation, retrieval-augmented generation and token-cost trade-offs, imbalanced classification handling, and precision-versus-recall decision-making within the Machine Learning domain for a Data Scientist role.