Choose Labels and Losses for Multiple Engagement Outcomes

How would you represent labels and choose a loss when a model predicts several engagement outcomes or engagement levels? Explain the difference between mutually exclusive classes, independent labels, and outcomes that follow a conditional funnel. ### Constraints & Assumptions - The source names multi-label, multi-engagement-level modeling, and loss functions without specifying exact outcomes. Use any example outcomes only as illustrative choices. - Clarify which outcomes can coexist and which labels are observable before selecting sigmoid, softmax, or a multi-task structure. - Include missing labels, imbalance, sampling, and metric interpretation. ### Clarifying Questions to Ask - Can one example have multiple positive outcomes, or exactly one class? - Are outcomes ordered, nested, or conditionally observed after an earlier event? - Are absent labels true negatives or missing observations? ```hint Label semantics determine the probability model Two outcomes that can both be true should not be forced to compete for one unit of softmax probability merely because they share a model. ``` ### What a Strong Answer Covers - Softmax cross-entropy for exclusive classes and sigmoid binary cross-entropy for multiple labels. - Multi-task loss weighting and masks for unobserved targets. - A distinction between marginal outcome probabilities and conditional funnel probabilities. - Effects of imbalance and negative sampling on training and calibration. - Per-task metrics and validation aligned with the final decision. ### Follow-up Questions - How would you stop a frequent easy task from dominating a rare important task? - Why can a model rank examples well but still produce poorly calibrated probabilities after negative sampling?