Logistic Regression from First Principles

Assumptions and Notation

• Binary classification with labels y ∈ {0, 1} and features x ∈ R^d.
• Linear score z = wᵀx + b; probability p(x) = P(y=1 | x).

Tasks

1. Derive logistic regression starting from a Bernoulli likelihood and the logit link.
2. Derive the negative log-likelihood (log-loss) and gradients for w and b.
3. Explain the logit link function and how decision thresholds relate to costs.
4. Discuss probability calibration: when it works, how to measure it, and how to fix miscalibration.
5. Compare L1 vs L2 regularization (optimization, sparsity, correlation, calibration).
6. Explain effects of feature scaling and modeling interactions.
7. Describe strategies for handling class imbalance.
8. Choose appropriate evaluation metrics for imbalanced data.
9. Interpret coefficients via odds and odds ratios.
10. Identify common failure modes and guardrails.