Implement binary logistic regression from scratch with NumPy

Quick Overview

Implement binary logistic regression from scratch with NumPy: a fit method that runs gradient descent on the log loss, plus probability and label prediction. Tests vectorized gradients, numerical stability for extreme scores, L2 regularization, array shapes, and optimizer trade-offs.

Implement binary logistic regression from scratch with NumPy

Company: Yahoo

Role: Machine Learning Engineer

Category: Machine Learning

Difficulty: easy

Interview Round: Onsite

Implement a binary logistic regression classifier from scratch in Python with NumPy, as you would by hand in a shared coding pad during a phone screen. The model takes a feature matrix and 0/1 labels, learns a weight vector and a bias by minimizing the log loss with gradient descent, and then returns probabilities and class labels for new rows. No machine-learning library such as scikit-learn or PyTorch may be used; NumPy is allowed. The report says only that the task was to hand-write binary logistic regression with NumPy. The interface below is an assumed, typical shape; confirm it with the interviewer. ```python import numpy as np class LogisticRegression: def __init__(self, lr: float = 0.1, n_iters: int = 1000, l2: float = 0.0): ... def fit(self, X: np.ndarray, y: np.ndarray) -> "LogisticRegression": ... def predict_proba(self, X: np.ndarray) -> np.ndarray: ... # shape (n,), P(y = 1) def predict(self, X: np.ndarray, threshold: float = 0.5) -> np.ndarray: ... # shape (n,), 0 or 1 ``` ```hint Vectorize the gradient Write the loss for all rows at once and differentiate it with respect to each row's score; the weight gradient should come out as a single matrix product, with no Python loop over samples. ``` ```hint Extreme scores Ask what `np.exp` returns for a score of -1000 or +1000, and what taking `np.log` of the resulting probability does to the loss. ``` ### Constraints and Clarifications - `X` has shape `(n, d)` with real-valued features, and `y` has shape `(n,)` with values 0 or 1. - Full-batch or mini-batch gradient descent are both acceptable; say which you use and why. - If you add L2 regularization, the bias is not penalized. - The code should run without overflow warnings or `nan` losses for any finite input. ### Clarifying Questions - Are the features already scaled, or should `fit` standardize them? - Is L2 regularization wanted, and should training stop after a fixed number of iterations or when the loss stops improving? - Can I assume both classes appear in the training labels? - Is the decision threshold fixed at 0.5, or chosen by the caller? ### What a Strong Answer Covers - A correct, vectorized forward pass, log loss, and gradient for both the weights and the bias - A numerically stable sigmoid and loss for large positive and negative scores - A training loop with a clear stopping rule, optional L2 that leaves the bias out, and loss tracking - Explicit handling of array shapes, plus a quick correctness check on a toy dataset - Time and memory cost per iteration, and when plain gradient descent is the wrong optimizer ### Follow-up Questions - Why train with the log loss instead of the mean squared error between the sigmoid output and the label? - How would you extend the model to more than two classes? - What happens to the weights if the training data is linearly separable and there is no regularization? - How would you train when the data does not fit in memory?

Overview: Implement binary logistic regression from scratch with NumPy: a fit method that runs gradient descent on the log loss, plus probability and label prediction. Tests vectorized gradients, numerical stability for extreme scores, L2 regularization, array shapes, and optimizer trade-offs.

Read the full Yahoo Machine Learning Engineer interview experience this question came from

Yahoo logo
Yahoo
Nov 8, 2026
easyMachine Learning EngineerOnsiteMachine Learning
0
0

Implement a binary logistic regression classifier from scratch in Python with NumPy, as you would by hand in a shared coding pad during a phone screen. The model takes a feature matrix and 0/1 labels, learns a weight vector and a bias by minimizing the log loss with gradient descent, and then returns probabilities and class labels for new rows. No machine-learning library such as scikit-learn or PyTorch may be used; NumPy is allowed.

The report says only that the task was to hand-write binary logistic regression with NumPy. The interface below is an assumed, typical shape; confirm it with the interviewer.

import numpy as np


class LogisticRegression:
    def __init__(self, lr: float = 0.1, n_iters: int = 1000, l2: float = 0.0): ...

    def fit(self, X: np.ndarray, y: np.ndarray) -> "LogisticRegression": ...

    def predict_proba(self, X: np.ndarray) -> np.ndarray: ...  # shape (n,), P(y = 1)

    def predict(self, X: np.ndarray, threshold: float = 0.5) -> np.ndarray: ...  # shape (n,), 0 or 1

Constraints and Clarifications

  • X has shape (n, d) with real-valued features, and y has shape (n,) with values 0 or 1.
  • Full-batch or mini-batch gradient descent are both acceptable; say which you use and why.
  • If you add L2 regularization, the bias is not penalized.
  • The code should run without overflow warnings or nan losses for any finite input.

Clarifying Questions Guidance

  • Are the features already scaled, or should fit standardize them?
  • Is L2 regularization wanted, and should training stop after a fixed number of iterations or when the loss stops improving?
  • Can I assume both classes appear in the training labels?
  • Is the decision threshold fixed at 0.5, or chosen by the caller?

What a Strong Answer Covers Guidance

  • A correct, vectorized forward pass, log loss, and gradient for both the weights and the bias
  • A numerically stable sigmoid and loss for large positive and negative scores
  • A training loop with a clear stopping rule, optional L2 that leaves the bias out, and loss tracking
  • Explicit handling of array shapes, plus a quick correctness check on a toy dataset
  • Time and memory cost per iteration, and when plain gradient descent is the wrong optimizer

Follow-up Questions Guidance

  • Why train with the log loss instead of the mean squared error between the sigmoid output and the label?
  • How would you extend the model to more than two classes?
  • What happens to the weights if the training data is linearly separable and there is no regularization?
  • How would you train when the data does not fit in memory?