A binary classifier will support a real decision, and the model produces a probability score for each observation. Explain precision and recall, how they change as the decision threshold moves, and how you would choose an operating point when false positives and false negatives have different costs. ### Constraints & Assumptions - The positive class may be uncommon. - A downstream action is taken only when the score exceeds a selected threshold. - The business costs of mistakes can be discussed qualitatively if exact dollar values are unavailable. ### Clarifying Questions to Ask - What does the positive class represent, and what action follows a positive prediction? - What are the consequences of a false positive and a false negative? - Is there a fixed review, outreach, or intervention capacity? - Are probability calibration and performance consistency across user groups important? ### What a Strong Answer Covers - Correct definitions using the confusion matrix and an explanation of why neither metric is threshold independent. - The precision-recall trade-off, especially under class imbalance, without claiming that one metric is universally better. - Threshold selection based on expected decision value, a capacity constraint, or a required safety/coverage target. - Use of a validation or test set, confidence intervals, calibration checks, and relevant segment analysis. - Recognition that PR-AUC summarizes ranking across thresholds but does not itself choose the production threshold. ### Follow-up Questions 1. Why can accuracy look strong when recall for the positive class is poor? 2. When would a precision requirement be treated as a hard constraint? 3. How would you respond if two groups have different precision at the same threshold? 4. What changes if the score is well ranked but poorly calibrated?