Run EDA and train models while preventing overfitting
Company: Imc
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
You are given a tabular regression dataset \(\{(x^{(j)}, y^{(j)})\}_{j=1}^M\) with numeric and categorical features and a continuous target.
Describe (at a code-and-practice level) how you would:
1) Perform exploratory data analysis (EDA) to identify data issues and useful patterns.
2) Build a baseline linear regression model, including preprocessing steps.
3) Train a small neural network regressor.
4) Detect and mitigate overfitting for both models.
Be explicit about the train/validation/test protocol, metrics, and common pitfalls.
Quick Answer: This question evaluates proficiency in exploratory data analysis, feature preprocessing, baseline linear regression and small neural network training, as well as detecting and mitigating overfitting and selecting appropriate train/validation/test protocols and evaluation metrics for tabular regression.