A recommendation model must choose one item from a very large catalog. Training approximates the full softmax by sampling negative items from a nonuniform proposal distribution `q(item)`. ### Clarifying Questions to Ask - Is the intended objective the full-catalog softmax likelihood or a different contrastive objective? - Are negatives sampled independently, and can the positive item also appear in the negative sample? - Are item frequencies used as the proposal distribution? ### Part 1 — Explain the Log-Q Correction Explain why sampled logits are adjusted using the sampling probability, what form the correction takes, and what goes wrong when frequent items are oversampled without correction. #### What This Part Should Cover - The distinction between the target class distribution and the proposal distribution. - Subtracting a term proportional to `log q(item)` from a sampled item's logit, with assumptions about the estimator. - Bias, variance, and numerical implementation considerations. ### Part 2 — Explain Why MSE Is Usually a Poor Classification Loss Compare mean squared error on class targets with cross-entropy for classification. #### What This Part Should Cover - Probabilistic interpretation, gradient behavior, calibration, and the effect of a sigmoid or softmax link. - Cases where a squared loss might still be a deliberate surrogate. ### What a Strong Answer Covers - Equations tied to the modeling assumptions rather than slogans. - A clear separation between sampled softmax and negative-sampling objectives. - Practical checks for sampling bugs and loss behavior. ### Follow-up Questions - How does the choice of `q` trade estimator variance against coverage? - What changes when negatives are sampled in-batch? - How would you test whether the correction is implemented with the right sign?