Compute Recall and FPR from Confusion Matrices to Select Classifiers

A multiple-choice question gives the confusion matrices of several binary classifiers, all evaluated on the same test set, and asks which classifiers meet a threshold on recall and a threshold on false positive rate (FPR). Compute recall and FPR for each classifier and select the ones that qualify. The report does not give the matrices or the thresholds, so the values below are illustrative. Rows are actual classes and columns are predicted classes; the test set has 80 positive and 120 negative examples. ```text predicted positive predicted negative A actual positive 68 12 actual negative 10 110 B actual positive 72 8 actual negative 18 102 C actual positive 60 20 actual negative 6 114 D actual positive 64 16 actual negative 12 108 ``` Select every classifier whose recall is at least 0.80 and whose FPR is at most 0.10. ```hint Which row each rate lives in Recall and FPR each use only one row of the matrix. Decide which row, and which cell of that row goes in the numerator. ``` ```hint Boundaries A classifier can land exactly on a threshold. Compare exact fractions rather than values rounded early. ``` ### Clarifying Questions - Is the positive class the first row and column in every matrix, or does the layout vary between options? - Are the thresholds inclusive ("at least 0.80", "at most 0.10") or strict? - Should the rates be reported as fractions or decimals, and to how many places? ### What a Strong Answer Covers - Correct definitions of recall (true positive rate) and FPR, each tied to the right row of the matrix - Correct values for every classifier and the selected set, with boundary cases settled by exact comparison - Distinguishes FPR and recall from easily confused quantities such as precision and the false discovery rate - Explains what the two thresholds trade off, and how moving the decision threshold changes both rates ### Follow-up Questions - More than one classifier passes. Which would you choose, and what more would you want to know before deciding? - Each classifier produces a score that is compared with a decision threshold. How does moving that threshold change recall and FPR, and how does this relate to the ROC curve? - Positives make up 1% of production traffic. Why can a classifier that passes both thresholds still produce mostly false alarms?