PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Machine Learning/Boston Consulting Group

Build and evaluate imbalanced binary classifier

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's skills in building reproducible machine learning pipelines for imbalanced binary classification, covering temporal splitting to avoid leakage, class imbalance handling, feature preprocessing, probability calibration, threshold selection, and monitoring for calibration drift.

  • medium
  • Boston Consulting Group
  • Machine Learning
  • Data Scientist

Build and evaluate imbalanced binary classifier

Company: Boston Consulting Group

Role: Data Scientist

Category: Machine Learning

Difficulty: medium

Interview Round: Take-home Project

You are given a binary classification dataset with severe class imbalance (positive rate ≈1%). Each row has: id, event_date (YYYY-MM-DD), categorical: region ∈ {NA, EU, APAC, LATAM, MEA}, and numerical features f1…f50. Labels are y ∈ {0,1}. Tasks: a) Build a reproducible training pipeline that: - splits temporally into train (≤2025-06-01), validation (2025-06-02–2025-08-01), and test (2025-08-02–2025-09-01); - applies standardization to numeric features and one-hot encoding to region; - handles imbalance inside CV folds (e.g., class_weight='balanced', or SMOTE within each fold without leaking validation data); - trains a strong baseline (e.g., calibrated logistic regression or gradient boosting) and outputs well-calibrated probabilities (Platt or isotonic on validation). b) Report ROC-AUC and PR-AUC on the test split; also report recall at 5% FPR and the decision threshold that maximizes F1 under the constraint recall ≥ 0.90. c) Describe how you would choose the operating point for a production system with a hard requirement of at most 2 false positives per 1,000 predictions. d) Discuss how calibration might drift over time and one technique to monitor and re-calibrate without label leakage.

Quick Answer: This question evaluates a data scientist's skills in building reproducible machine learning pipelines for imbalanced binary classification, covering temporal splitting to avoid leakage, class imbalance handling, feature preprocessing, probability calibration, threshold selection, and monitoring for calibration drift.

Related Interview Questions

  • Design and sample for credit default prediction - Boston Consulting Group (Medium)
  • Explain AUC, imbalance, losses, and networks - Boston Consulting Group (medium)
  • Reduce overfitting under constraints - Boston Consulting Group (hard)
  • Achieve 0.95 precision via thresholding - Boston Consulting Group (medium)
  • Build a leak-free sklearn pipeline - Boston Consulting Group (medium)
Boston Consulting Group logo
Boston Consulting Group
Oct 13, 2025, 9:49 PM
Data Scientist
Take-home Project
Machine Learning
2
0

Take‑home: Imbalanced Binary Classification with Temporal Split, Calibration, and Operating Point Selection

Context

You are given an event‑level dataset for a binary classification problem with severe class imbalance (positive rate ≈ 1%). The goal is to build a reproducible modeling pipeline, evaluate it with appropriate metrics, and propose a principled operating point for production.

Data

  • Columns per row:
    • id
    • event_date (YYYY-MM-DD)
    • region ∈ {NA, EU, APAC, LATAM, MEA}
    • Numerical features f1, f2, …, f50
    • Label y ∈ {0, 1}

Temporal Splits (no leakage)

  • Train: event_date ≤ 2025-06-01
  • Validation: 2025-06-02 – 2025-08-01
  • Test: 2025-08-02 – 2025-09-01

Tasks

(a) Build a reproducible training pipeline that:

  1. Performs the temporal split as specified.
  2. Applies StandardScaler to numeric features and OneHotEncoder to region.
  3. Handles class imbalance inside CV folds (e.g., class_weight='balanced' or SMOTE/SMOTENC within each fold without leaking validation data).
  4. Trains a strong baseline (e.g., calibrated logistic regression or gradient boosting) and outputs well‑calibrated probabilities (Platt/sigmoid or isotonic, calibrated on the validation split).

(b) Report on the test split:

  • ROC-AUC and PR-AUC
  • Recall at 5% FPR
  • The decision threshold that maximizes F1 subject to recall ≥ 0.90 (select the threshold using validation, then report the chosen value and resulting test performance)

(c) Describe how to choose the operating point for a production system with a hard requirement of at most 2 false positives per 1,000 predictions.

(d) Discuss how calibration might drift over time and one technique to monitor and re‑calibrate without label leakage.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Machine Learning•More Boston Consulting Group•More Data Scientist•Boston Consulting Group Data Scientist•Boston Consulting Group Machine Learning•Data Scientist Machine Learning
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.