Estimate OLS via streaming sufficient statistics

Read the full interview experience this question came from →

Quick Overview

This question evaluates proficiency in streaming/out-of-core linear regression, including computing sufficient statistics with an intercept, assessing numerical stability of normal equations versus QR/SVD or incremental methods, incorporating ridge penalties, and designing parallel fault-tolerant computations.

Estimate OLS via streaming sufficient statistics

Company: Citadel

Role: Data Scientist

Category: Machine Learning

Difficulty: hard

Interview Round: Technical Screen

You must estimate OLS coefficients β for very high-dimensional linear regression with data too large to fit in memory. (1) Derive how to compute XᵀX and Xᵀy in streaming mini-batches (include an intercept), then recover β and standard errors. (2) Discuss numerical stability vs. using QR or incremental/online methods. (3) Extend to ridge regression and show how to update with λI. (4) Explain how you would checkpoint and parallelize the computation.

Overview: This question evaluates proficiency in streaming/out-of-core linear regression, including computing sufficient statistics with an intercept, assessing numerical stability of normal equations versus QR/SVD or incremental methods, incorporating ridge penalties, and designing parallel fault-tolerant computations.

Read the full Citadel Data Scientist interview experience this question came from

|Home/Machine Learning/Citadel
Citadel logo
Citadel
Oct 13, 2025
hardData ScientistTechnical ScreenMachine Learning
17
0

Streaming OLS and Ridge for Out-of-Core, High-Dimensional Linear Regression

You need to estimate linear regression coefficients when the dataset is too large to fit in memory. Assume we can read data in mini-batches of rows. Let X ∈ R^{n×p} be the feature matrix and y ∈ R^{n} the target. Include an intercept.

  1. Show how to compute the sufficient statistics XᵀX and Xᵀy in streaming mini-batches (with an intercept), then recover β and standard errors.
  2. Discuss numerical stability of using the normal equations vs. more stable QR/SVD or incremental/online methods.
  3. Extend to ridge regression and show how to incorporate the λI penalty in the out-of-core computation.
  4. Explain how you would checkpoint for fault tolerance and parallelize the computation across workers.
Loading comments...

Browse More Questions

More Machine Learning•More Citadel•More Data Scientist•Citadel Data Scientist•Citadel Machine Learning•Data Scientist Machine Learning