PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Machine Learning/Microsoft

How would you build and evaluate a classifier?

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's proficiency in binary classification model evaluation, end-to-end machine learning project design, and model interpretability, covering confusion matrix interpretation, the implications of Type I and Type II errors, trade-offs among metrics (accuracy, precision, recall, specificity, F1, ROC-AUC, PR-AUC, calibration), thresholding, deployment, monitoring, and SHAP-based explanations and their limitations. It is commonly asked in Machine Learning interviews for Data Scientist roles because it probes both conceptual understanding and practical application—assessing how metric choice and operational decisions align with business costs, class imbalance, and downstream actions.

  • medium
  • Microsoft
  • Machine Learning
  • Data Scientist

How would you build and evaluate a classifier?

Company: Microsoft

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Technical Screen

You are building a binary classification model for a business use case such as fraud detection, churn prediction, lead scoring, or content moderation. Explain how you would evaluate the model using the confusion matrix, and clarify the meaning of Type I and Type II errors in this setting. Discuss how metric choice should depend on business costs, class imbalance, and the downstream action taken on the model score. Compare metrics such as accuracy, precision, recall, specificity, F1, ROC-AUC, PR-AUC, and calibration. Then describe how you would build the machine learning project end to end: problem framing, label definition, train/validation/test strategy, feature engineering, model selection, thresholding, error analysis, deployment, and monitoring. Finally, explain how you would interpret SHAP analysis, what it is useful for, and its limitations.

Quick Answer: This question evaluates a data scientist's proficiency in binary classification model evaluation, end-to-end machine learning project design, and model interpretability, covering confusion matrix interpretation, the implications of Type I and Type II errors, trade-offs among metrics (accuracy, precision, recall, specificity, F1, ROC-AUC, PR-AUC, calibration), thresholding, deployment, monitoring, and SHAP-based explanations and their limitations. It is commonly asked in Machine Learning interviews for Data Scientist roles because it probes both conceptual understanding and practical application—assessing how metric choice and operational decisions align with business costs, class imbalance, and downstream actions.

Related Interview Questions

  • How do you choose a model? - Microsoft (medium)
  • Explain SHAP in an ML System - Microsoft (medium)
  • Explain normalization, regularization, CTR, imbalance handling - Microsoft (medium)
  • Clean OCR data and build an LLM dataset - Microsoft (medium)
  • Explain SHAP and build an ML project - Microsoft (easy)
Microsoft logo
Microsoft
Jan 16, 2026, 12:00 AM
Data Scientist
Technical Screen
Machine Learning
1
0
Loading...

You are building a binary classification model for a business use case such as fraud detection, churn prediction, lead scoring, or content moderation.

Explain how you would evaluate the model using the confusion matrix, and clarify the meaning of Type I and Type II errors in this setting. Discuss how metric choice should depend on business costs, class imbalance, and the downstream action taken on the model score. Compare metrics such as accuracy, precision, recall, specificity, F1, ROC-AUC, PR-AUC, and calibration.

Then describe how you would build the machine learning project end to end: problem framing, label definition, train/validation/test strategy, feature engineering, model selection, thresholding, error analysis, deployment, and monitoring. Finally, explain how you would interpret SHAP analysis, what it is useful for, and its limitations.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Machine Learning•More Microsoft•More Data Scientist•Microsoft Data Scientist•Microsoft Machine Learning•Data Scientist Machine Learning
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.