##### Scenario Building a linear regression model from scratch; parameters are optimized through batch gradient descent. ##### Question Write Python-style pseudocode for batch gradient descent that minimizes mean-squared error for linear regression. Explain briefly what each step in your pseudocode does. ##### Hints Cover initialization, prediction, error, gradient, parameter update, loop/stop conditions.

Amazon data scientist machine learning prompt on writing batch gradient descent pseudocode for linear regression, including MSE loss, vectorized gradients, parameter updates, stopping criteria, and implementation checks.

How do I approach Machine Learning interview questions?

Machine Learning questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master machine learning interviews.

What difficulty level is this interview question?

This is a easy difficulty Machine Learning question, commonly asked during Technical Screen rounds at Amazon.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Amazon during technical interviews.

Implement Batch Gradient Descent for Linear Regression

Batch Gradient Descent for Linear Regression

You are building a linear regression model from scratch and will optimize the parameters using batch gradient descent.

Assume:

X is an n x d feature matrix.
y is an n -length target vector.
w is a d -length weight vector.
b is a scalar intercept.
The objective is mean squared error.

Constraints & Assumptions

Write Python-style pseudocode, not a library call to scikit-learn.
Use vectorized operations where possible.
Include initialization, prediction, loss, gradients, updates, and stopping criteria.
Explain each major step briefly.
Mention practical issues such as feature scaling, learning rate, and convergence.

Clarifying Questions to Ask Guidance

Should the intercept be modeled as a separate scalar or as a column of ones in X ?
Should I include regularization?
Is the interviewer expecting full Python code or high-level pseudocode?
What stopping criterion should be used: fixed iterations, loss tolerance, or gradient norm?

What a Strong Answer Covers Guidance

Prediction formula y_hat = X @ w + b .
MSE objective and error vector error = y_hat - y .
Correct batch gradients: grad_w = (2/n) * X.T @ error and grad_b = (2/n) * sum(error) .
Parameter updates: w = w - lr * grad_w , b = b - lr * grad_b .
A loop over iterations with loss tracking and convergence checks.
Notes on feature scaling, choosing learning rate, avoiding divergence, and validating with a closed-form or library baseline.

Follow-up Questions Guidance

How would the update change with L2 regularization?
Why might gradient descent diverge?
How would mini-batch gradient descent differ from batch gradient descent?
How would you test that your implementation is correct?

Batch Gradient Descent for Linear Regression

You are building a linear regression model from scratch and will optimize the parameters using batch gradient descent.

Assume:

X is an n x d feature matrix.

y is an n -length target vector.

w is a d -length weight vector.

b is a scalar intercept.

The objective is mean squared error.

Constraints & Assumptions

Write Python-style pseudocode, not a library call to scikit-learn.

Use vectorized operations where possible.

Include initialization, prediction, loss, gradients, updates, and stopping criteria.

Explain each major step briefly.

Mention practical issues such as feature scaling, learning rate, and convergence.

Clarifying Questions to Ask Guidance

Should the intercept be modeled as a separate scalar or as a column of ones in X ?

Should I include regularization?

Is the interviewer expecting full Python code or high-level pseudocode?

What stopping criterion should be used: fixed iterations, loss tolerance, or gradient norm?

What a Strong Answer Covers Guidance

Prediction formula y_hat = X @ w + b .

MSE objective and error vector error = y_hat - y .

Correct batch gradients: grad_w = (2/n) * X.T @ error and grad_b = (2/n) * sum(error) .

Parameter updates: w = w - lr * grad_w , b = b - lr * grad_b .

A loop over iterations with loss tracking and convergence checks.

Notes on feature scaling, choosing learning rate, avoiding divergence, and validating with a closed-form or library baseline.

Follow-up Questions Guidance

How would the update change with L2 regularization?

Why might gradient descent diverge?

How would mini-batch gradient descent differ from batch gradient descent?

How would you test that your implementation is correct?

Implement Batch Gradient Descent for Linear Regression

Quick Overview

Implement Batch Gradient Descent for Linear Regression

Batch Gradient Descent for Linear Regression

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer

Implement Batch Gradient Descent for Linear Regression

Quick Overview

Implement Batch Gradient Descent for Linear Regression

Batch Gradient Descent for Linear Regression

Constraints & Assumptions

Clarifying Questions to Ask Guidance

What a Strong Answer Covers Guidance

Follow-up Questions Guidance

Write your answer