Prove OLS invariance to linear transforms

Read the full interview experience this question came from →

Quick Overview

This question evaluates understanding of linear regression invariance under invertible linear transformations, multivariate linear algebra, and the impact of regularization (ridge and lasso) on coefficient and prediction stability in the Statistics & Math domain.

Prove OLS invariance to linear transforms

Company: Google

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Technical Screen

You fit Model 1: y ~ X1 + X2. You also fit Model 2 using Z = [X1 − X2, X1 + X2] = X T where T = [[1,1], [−1,1]] (2×2, invertible). a) Prove that OLS predictions ŷ are identical for Model 1 and Model 2 for any invertible T; derive the mapping between coefficients (b_Z = T^{-1} b_X). b) If you use ridge (λ||b||₂²) or lasso (λ||b||₁) instead of OLS, will coefficients and predictions remain invariant under this T? Specify conditions precisely (e.g., ridge invariance to orthonormal transforms but not arbitrary scalings; lasso invariance only to signed permutations, not general rotations). c) For ridge, write solutions and show when ŷ is unchanged; for lasso, provide a concrete counterexample where predictions differ.

Overview: This question evaluates understanding of linear regression invariance under invertible linear transformations, multivariate linear algebra, and the impact of regularization (ridge and lasso) on coefficient and prediction stability in the Statistics & Math domain.

Read the full Google Data Scientist interview experience this question came from

|Home/Statistics & Math/Google
Google logo
Google
Oct 13, 2025
mediumData ScientistTechnical ScreenStatistics & Math
14
0

You fit Model 1: y ~ X1 + X2. You also fit Model 2 using Z = [X1 − X2, X1 + X2] = X T where T = [[1,1], [−1,1]] (2×2, invertible). a) Prove that OLS predictions ŷ are identical for Model 1 and Model 2 for any invertible T; derive the mapping between coefficients (b_Z = T^{-1} b_X). b) If you use ridge (λ||b||₂²) or lasso (λ||b||₁) instead of OLS, will coefficients and predictions remain invariant under this T? Specify conditions precisely (e.g., ridge invariance to orthonormal transforms but not arbitrary scalings; lasso invariance only to signed permutations, not general rotations). c) For ridge, write solutions and show when ŷ is unchanged; for lasso, provide a concrete counterexample where predictions differ.

Loading comments...