Evaluate Noisy Data for LLM Post-Training
Company: Mercor
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
You are working on post-training a large language model. You receive a large, noisy dataset collected from multiple sources.
Discuss how you would:
1. Decide which data is suitable for LLM post-training.
2. Filter and prioritize the dataset before training.
3. Design ablation studies to determine whether this dataset is useful.
4. Run post-training effectively while avoiding regressions in existing model quality.
Your answer should cover data quality criteria, experimental design, metrics, controls, and practical post-training considerations.
Quick Answer: This question evaluates competency in data curation, noisy-data assessment, post-training procedures for large language models, experimental design (including ablation studies and control conditions), and evaluation metrics for detecting regressions.