Derive L1 vs L2 effects with correlation

Q: Derive L1 vs L2 effects with correlation

This question evaluates understanding of regularization methods (ridge/L2, LASSO/L1, and elastic net), multicollinearity effects, geometric intuition of constraint sets, and hyperparameter influence on coefficient patterns in the Statistics & Math domain.

Q: How do I approach Statistics & Math interview questions?

Statistics & Math questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master statistics & math interviews.

Question

Multicollinearity: Ridge vs LASSO vs Elastic Net

Setup

Two standardized predictors x1 and x2 with corr(x1, x2) = 0.99.
X'X = [[100, 99], [99, 100]] and X'y = [120, 120].

Tasks

Compute the ridge (L2) estimate w_ridge = (X'X + λI)^{-1} X'y for λ = 10. Report numeric weights to 2 decimal places and explain why L2 prefers sharing weight under high collinearity.
Without solving the full LASSO, argue which coefficient pattern the L1 solution is likely to produce for λ1 = 10: both similar and small, one near zero and the other large, or something else? Justify using the geometry of L1 vs L2 constraint sets under high collinearity.
Propose elastic-net penalties (α and λ) that would stabilize selection while controlling variance. Explain how you would tune α and λ and which validation metric you would pick if the goal is sparse interpretation with minimal loss in PR-AUC.

Derive L1 vs L2 effects with correlation

Multicollinearity: Ridge vs LASSO vs Elastic Net

Setup

Tasks

Solution

Comments (0)

Derive L1 vs L2 effects with correlation

Overview

Multicollinearity: Ridge vs LASSO vs Elastic Net

Setup

Tasks

Solution

Comments (0)