How would you choose a classification threshold?
Company: Bytedance
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Technical Screen
You trained a binary classifier that outputs a probability score p(y=1|x). You must choose a decision threshold t to convert probabilities into class labels.
Answer the following:
- What is your general approach to selecting t on a validation set?
- How do you incorporate different costs of false positives vs false negatives?
- How do class imbalance and probability calibration affect threshold choice?
- Give one example where you would prefer a high threshold and one where you would prefer a low threshold.
Quick Answer: This question evaluates understanding of threshold selection for probabilistic binary classifiers, testing competencies in validation-set decision threshold choice, cost-sensitive trade-offs between false positives and false negatives, effects of class imbalance, and probability calibration.