Implement Batch Gradient Descent for Linear Regression

Quick Overview

Amazon data scientist machine learning prompt on writing batch gradient descent pseudocode for linear regression, including MSE loss, vectorized gradients, parameter updates, stopping criteria, and implementation checks.

Implement Batch Gradient Descent for Linear Regression

Company: Amazon

Role: Data Scientist

Category: Machine Learning

Difficulty: easy

Interview Round: Technical Screen

##### Scenario Building a linear regression model from scratch; parameters are optimized through batch gradient descent. ##### Question Write Python-style pseudocode for batch gradient descent that minimizes mean-squared error for linear regression. Explain briefly what each step in your pseudocode does. ##### Hints Cover initialization, prediction, error, gradient, parameter update, loop/stop conditions.

Overview: Amazon data scientist machine learning prompt on writing batch gradient descent pseudocode for linear regression, including MSE loss, vectorized gradients, parameter updates, stopping criteria, and implementation checks.

|Home/Machine Learning/Amazon
Amazon logo
Amazon
Jul 12, 2025
easyData ScientistTechnical ScreenMachine Learning
21
0

Batch Gradient Descent for Linear Regression

You are building a linear regression model from scratch and will optimize the parameters using batch gradient descent.

Assume:

  • X is an n x d feature matrix.
  • y is an n -length target vector.
  • w is a d -length weight vector.
  • b is a scalar intercept.
  • The objective is mean squared error.

Constraints & Assumptions

  • Write Python-style pseudocode, not a library call to scikit-learn.
  • Use vectorized operations where possible.
  • Include initialization, prediction, loss, gradients, updates, and stopping criteria.
  • Explain each major step briefly.
  • Mention practical issues such as feature scaling, learning rate, and convergence.

Clarifying Questions to Ask Guidance

  • Should the intercept be modeled as a separate scalar or as a column of ones in X ?
  • Should I include regularization?
  • Is the interviewer expecting full Python code or high-level pseudocode?
  • What stopping criterion should be used: fixed iterations, loss tolerance, or gradient norm?

What a Strong Answer Covers Guidance

  • Prediction formula y_hat = X @ w + b .
  • MSE objective and error vector error = y_hat - y .
  • Correct batch gradients: grad_w = (2/n) * X.T @ error and grad_b = (2/n) * sum(error) .
  • Parameter updates: w = w - lr * grad_w , b = b - lr * grad_b .
  • A loop over iterations with loss tracking and convergence checks.
  • Notes on feature scaling, choosing learning rate, avoiding divergence, and validating with a closed-form or library baseline.

Follow-up Questions Guidance

  • How would the update change with L2 regularization?
  • Why might gradient descent diverge?
  • How would mini-batch gradient descent differ from batch gradient descent?
  • How would you test that your implementation is correct?
Loading comments...

Browse More Questions

More Machine Learning•More Amazon•More Data Scientist•Amazon Data Scientist•Amazon Machine Learning•Data Scientist Machine Learning