Derive logistic regression objective and gradients

Quick Overview

This question evaluates understanding of logistic regression, maximum likelihood estimation, cross-entropy (negative log-likelihood), L2 regularization, and the ability to compute model gradients for binary classification.

Derive logistic regression objective and gradients

Company: Upstart

Role: Data Scientist

Category: Machine Learning

Difficulty: easy

Interview Round: HR Screen

Binary logistic regression with dataset {(x_i, y_i)}_{i=1}^m, y_i ∈ {0,1}. Using the sigmoid σ(z)=1/(1+e^{-z}) and linear score z_i = w^T x_i + b: a) Write the exact maximum-likelihood optimization objective (state clearly whether it is a maximization or minimization) both without and with L2 regularization of strength λ≥0. b) Write the explicit negative log-likelihood (cross-entropy) L(w,b). c) Derive the gradients ∂L/∂w and ∂L/∂b. d) Clarify the distinction among 'objective', 'loss', and 'regularized objective' in this context.

Quick Answer: This question evaluates understanding of logistic regression, maximum likelihood estimation, cross-entropy (negative log-likelihood), L2 regularization, and the ability to compute model gradients for binary classification.

|Home/Machine Learning/Upstart
Upstart logo
Upstart
Oct 13, 2025, 9:49 PM
easyData ScientistHR ScreenMachine Learning
4
0

Context: Binary Logistic Regression

You are given a binary classification dataset {(x_i, y_i)}_{i=1}^m with labels y_i ∈ {0, 1}. The model uses the sigmoid function σ(z) = 1/(1 + e^{-z}) and a linear score z_i = w^T x_i + b.

Answer the following:

(a) Write the exact maximum-likelihood optimization objective (state clearly whether it is a maximization or a minimization) both without and with L2 regularization of strength λ ≥ 0.

(b) Write the explicit negative log-likelihood (cross-entropy) L(w, b).

(c) Derive the gradients ∂L/∂w and ∂L/∂b.

(d) Clarify the distinction among "objective", "loss", and "regularized objective" in this context.

Loading comments...