Explain Linear Regression Assumptions, Evaluation, and Feature Addition

Explain the problem solved by linear regression, how the model is fit, which assumptions matter, and how you would evaluate its quality. Then explain what happens to R-squared when new features are added. ### Part 1 — Fit and Interpret the Model State a linear-regression objective and describe how to estimate its coefficients. #### What This Part Should Cover - The response, predictor matrix, intercept, and least-squares objective. - A numerically appropriate fitting method and the role of rank or collinearity. - The difference between assumptions for a useful conditional-mean model, coefficient inference, and prediction. ### Part 2 — Evaluate Regression Quality Choose evaluation metrics and diagnostics and explain what they reveal. #### What This Part Should Cover - Error measures and R-squared on an appropriate validation split. - Residual patterns, outliers, and limitations of relying on one metric. - The difference between fitting the observed data and generalizing to new observations. ### Part 3 — Add Predictors Does adding features make R-squared higher, lower, unchanged, or does it depend? #### What This Part Should Cover - The exact fitting and evaluation conditions under which training R-squared cannot decrease. - Why the enlarged model includes the old model as a special case. - The different behavior of adjusted R-squared and out-of-sample R-squared. ### Clarifying Questions - Are we discussing unregularized least squares with an intercept on the same observations? - Is the metric measured on training data or held-out data? - Is the goal prediction, interpretation of coefficients, or causal interpretation? ```hint Compare the two feasible model sets After adding a predictor, consider whether a coefficient choice can reproduce every prediction made by the original model. ``` ### What a Strong Answer Covers The explanation separates the fitting objective from inference assumptions and evaluation. It states the conditions of the R-squared result precisely and does not assume that predictors or the marginal target must be normally distributed. ### Follow-up Questions 1. Why might adding a feature improve training R-squared while worsening test performance? 2. What happens when two predictor columns are exactly redundant? 3. When are normally distributed errors relevant, and when are they unnecessary for computing the least-squares fit?