## Prompt Review a binary logistic-regression implementation whose prediction is `h = sigmoid(X @ w)`. The code computes `grad = X.T @ h / m`, defines `sigmoid_derivative(z)` as `z * (1-z)` even when callers pass a pre-activation, and computes cost using only `-y * log(h)`. Identify and correct the mathematical bugs, derive the gradient, and describe numerical and testing safeguards. ### Constraints & Assumptions - `X` has shape `(m, d)`, `w` has shape `(d,)`, and `y` contains zeros and ones. - The loss is mean binary cross-entropy without regularization. - The implementation should remain stable for very large positive or negative logits. - A correct answer must distinguish a pre-activation `z` from an already-sigmoided probability `h`. ### Clarifying Questions to Ask - Is an intercept already included as a column of ones in `X`? - Does the training loop need an explicit sigmoid derivative when differentiating cross-entropy with sigmoid? - Should cost be computed from probabilities or directly from logits? ```hint Simplify the composed derivative For sigmoid followed by binary cross-entropy, the probability-denominator terms cancel the sigmoid derivative, leaving `h-y` at the logit. ``` ```hint Test gradients independently Compare the analytic gradient with central finite differences on a tiny non-symmetric dataset. ``` ### What a Strong Answer Covers - Complete binary cross-entropy with both positive and negative class terms. - A clear chain-rule derivation ending at `X.T @ (h-y) / m`. - Correct definitions for sigmoid derivative under both possible inputs. - Stable log-loss or softplus computation and shape checks. - Finite-difference gradient tests and targeted cases for all-zero/all-one labels and extreme logits. ### Follow-up Questions 1. How does L2 regularization change the cost and gradient? 2. Why can clipping probabilities hide instability that a logits-based loss avoids? 3. How would class weighting alter the simple `h-y` expression?