Contrast L1 and L2 regularization effects

Q: Contrast L1 and L2 regularization effects

This question evaluates a candidate's understanding of regularization techniques in supervised learning, covering the geometric intuition behind L1 versus L2 penalties, sparsity and bias–variance trade-offs, behavior under multicollinearity and elastic net considerations, algorithmic solution differences, and the penalized logistic regression objective and tuning. It is commonly asked in the Machine Learning domain to assess both conceptual understanding and practical application of model regularization, numerical stability, and reproducible hyperparameter selection, and the expected level spans conceptual reasoning and implementation-aware details.

Q: How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty Machine Learning question, commonly asked during Onsite rounds.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates during technical interviews.

Question

Ridge (L2) vs Lasso (L1) in Linear and Logistic Regression

Context: You are comparing L2 (Ridge) and L1 (Lasso) regularization for linear and logistic regression. Assume predictors are columns of X, response is y, and the intercept is not penalized unless stated. We write optimization in penalized form (minimize loss + penalty).

(a) Shrinkage geometry and sparsity

Explain the geometric intuition of L1 vs L2 constraints/penalties and why L1 tends to yield exact zeros (sparse solutions), while L2 rarely does.

(b) Bias of penalized estimates

Are L1/L2 penalized estimators unbiased? Provide a derivation or counterexample and discuss the bias–variance trade-off.

(c) Multicollinearity

Under strong multicollinearity, which penalty is preferable and why? Briefly mention when elastic net might be better.

(d) Algorithms and library differences

Describe coordinate descent and solution path algorithms (e.g., LARS). Explain why solutions can differ across libraries due to: penalty scaling choices, standardization/centering, intercept treatment, and λ-grid construction and tolerance.

(e) Logistic regression with L2

Write the penalized (negative) log-likelihood to minimize and its gradient for L2-penalized logistic regression (intercept unpenalized). Describe how you would select λ using nested cross-validation, including key guardrails to avoid leakage.

Contrast L1 and L2 regularization effects

Quick Overview

Contrast L1 and L2 regularization effects

Ridge (L2) vs Lasso (L1) in Linear and Logistic Regression

Write your answer