Derive and implement calibration via temperature scaling

Quick Overview

This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Derive and implement calibration via temperature scaling states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Derive and implement calibration via temperature scaling

Company: NewsBreak

Role: Machine Learning Engineer

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

Given a trained softmax classifier with logits z, derive temperature scaling for probability calibration: define p_i(x; T) = softmax(z_i(x) / T). Formulate the negative log-likelihood on a held-out validation set and derive the gradient with respect to T. Then write Python code to learn T by minimizing this objective (e.g., with gradient descent) and a function that applies the learned T to calibrate new predictions.

Quick Answer: This interview question evaluates core ML concepts, assumptions, math intuition, training/evaluation trade-offs, and practical failure modes in a realistic interview setting. A strong answer for Derive and implement calibration via temperature scaling states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/Machine Learning/NewsBreak
NewsBreak logo
NewsBreak
Aug 9, 2025, 12:00 AM
mediumMachine Learning EngineerTechnical ScreenMachine Learning
3
0

Derive and implement calibration via temperature scaling

Temperature Scaling for Softmax Calibration

Context

You have a trained multi-class classifier that outputs logits z(x) ∈ R^K for input x (the classifier is fixed; only calibration is learned). Temperature scaling calibrates predicted probabilities as:

p_i(x; T) = softmax(z_i(x) / T)

where T > 0 is a single scalar temperature shared across classes and inputs.

You are given a held-out validation set with logits and true labels, and you must learn T by minimizing negative log-likelihood (NLL).

Task

  1. Write the NLL on the validation set as a function of T.
  2. Derive the gradient of this NLL with respect to T.
  3. Implement Python code that learns T by minimizing this NLL (e.g., gradient descent), and provide a function that applies the learned T to calibrate new predictions.

Constraints & Assumptions

  • Preserve the scope, facts, inputs, and requested outputs from the prompt above.
  • If the prompt leaves a detail unspecified, state a reasonable assumption before relying on it.
  • Keep the answer interview-ready: concise enough to present, but concrete enough to implement or evaluate.

Clarifying Questions to Ask Guidance

  • Clarify the task, data shape, labels, constraints, and evaluation metric.
  • State assumptions behind the math or modeling technique you choose.
  • Connect theory to practical training, debugging, and deployment implications.

What a Strong Answer Covers Guidance

  • Correct definitions and formulas where the prompt requires them.
  • A practical explanation of how the method behaves on real data.
  • Trade-offs, failure modes, diagnostics, and mitigation strategies.
  • Evaluation choices that match the product or modeling objective.

Follow-up Questions Guidance

  • How would noisy labels, class imbalance, or distribution shift affect the answer?
  • What would you monitor after deployment?
  • Which baseline would you compare against first?
Loading comments...

Browse More Questions

More Machine Learning•More NewsBreak•More Machine Learning Engineer•NewsBreak Machine Learning Engineer•NewsBreak Machine Learning•Machine Learning Engineer Machine Learning