Implement binary logistic regression from scratch with NumPy
Company: Yahoo
Role: Machine Learning Engineer
Category: Machine Learning
Difficulty: easy
Interview Round: Onsite
Implement a binary logistic regression classifier from scratch in Python with NumPy, as you would by hand in a shared coding pad during a phone screen. The model takes a feature matrix and 0/1 labels, learns a weight vector and a bias by minimizing the log loss with gradient descent, and then returns probabilities and class labels for new rows. No machine-learning library such as scikit-learn or PyTorch may be used; NumPy is allowed.
The report says only that the task was to hand-write binary logistic regression with NumPy. The interface below is an assumed, typical shape; confirm it with the interviewer.
```python
import numpy as np
class LogisticRegression:
def __init__(self, lr: float = 0.1, n_iters: int = 1000, l2: float = 0.0): ...
def fit(self, X: np.ndarray, y: np.ndarray) -> "LogisticRegression": ...
def predict_proba(self, X: np.ndarray) -> np.ndarray: ... # shape (n,), P(y = 1)
def predict(self, X: np.ndarray, threshold: float = 0.5) -> np.ndarray: ... # shape (n,), 0 or 1
```
```hint Vectorize the gradient
Write the loss for all rows at once and differentiate it with respect to each row's score; the weight gradient should come out as a single matrix product, with no Python loop over samples.
```
```hint Extreme scores
Ask what `np.exp` returns for a score of -1000 or +1000, and what taking `np.log` of the resulting probability does to the loss.
```
### Constraints and Clarifications
- `X` has shape `(n, d)` with real-valued features, and `y` has shape `(n,)` with values 0 or 1.
- Full-batch or mini-batch gradient descent are both acceptable; say which you use and why.
- If you add L2 regularization, the bias is not penalized.
- The code should run without overflow warnings or `nan` losses for any finite input.
### Clarifying Questions
- Are the features already scaled, or should `fit` standardize them?
- Is L2 regularization wanted, and should training stop after a fixed number of iterations or when the loss stops improving?
- Can I assume both classes appear in the training labels?
- Is the decision threshold fixed at 0.5, or chosen by the caller?
### What a Strong Answer Covers
- A correct, vectorized forward pass, log loss, and gradient for both the weights and the bias
- A numerically stable sigmoid and loss for large positive and negative scores
- A training loop with a clear stopping rule, optional L2 that leaves the bias out, and loss tracking
- Explicit handling of array shapes, plus a quick correctness check on a toy dataset
- Time and memory cost per iteration, and when plain gradient descent is the wrong optimizer
### Follow-up Questions
- Why train with the log loss instead of the mean squared error between the sigmoid output and the label?
- How would you extend the model to more than two classes?
- What happens to the weights if the training data is linearly separable and there is no regularization?
- How would you train when the data does not fit in memory?
Overview: Implement binary logistic regression from scratch with NumPy: a fit method that runs gradient descent on the log loss, plus probability and label prediction. Tests vectorized gradients, numerical stability for extreme scores, L2 regularization, array shapes, and optimizer trade-offs.
Read the full Yahoo Machine Learning Engineer interview experience this question came from