Choose regularization norms and model formulations

Regularization and model choice. 1) For linear and logistic regression, write the objective functions with L0, L1, L2, and L-infinity penalties in both constrained and penalized forms (e.g., minimize loss subject to norm constraint; and minimize loss + lambda * norm). For each norm, state expected coefficient patterns (sparsity, shrinkage, robustness to outliers) and optimization difficulty. 2) Given features X where two groups of predictors are highly collinear and a few outliers exist in X and y, justify which penalty (or elastic net mixing) you would choose and why. 3) Contrast linear regression and logistic regression by explicitly writing the model formulas (link function, conditional distributions) and stating when using linear regression for classification is inappropriate. 4) Discuss how feature scaling interacts with L1 and L2, and what happens if you omit scaling.