Build pipeline for imbalanced classification

Quick Overview

This interview question evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer for Build pipeline for imbalanced classification states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

Build pipeline for imbalanced classification

Company: DRW

Role: Machine Learning Engineer

Category: ML System Design

Difficulty: medium

Interview Round: Take-home Project

##### Question Using scikit-learn and imbalanced-learn, build a classification pipeline that handles severe class imbalance, performs standard preprocessing, applies an appropriate resampling method, trains a classifier, and outputs precision, recall, and F1-score on a held-out set.

Quick Answer: This interview question evaluates ML product requirements, data/labeling, modeling, serving architecture, evaluation, monitoring, and trade-offs in a realistic interview setting. A strong answer for Build pipeline for imbalanced classification states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.

|Home/ML System Design/DRW
DRW logo
DRW
Aug 4, 2025, 10:55 AM
mediumMachine Learning EngineerTake-home ProjectML System Design
4
0

Build pipeline for imbalanced classification

Build an Imbalanced Classification Pipeline (scikit-learn + imbalanced-learn)

Context

You are given a tabular dataset with a severely imbalanced binary target (e.g., minority class rate < 5%). Build an end-to-end classification pipeline that:

  • Applies standard preprocessing to numeric and categorical features.
  • Uses an appropriate resampling method to address imbalance.
  • Trains a classifier.
  • Evaluates precision, recall, and F1-score on a held-out test set.

Assume the input features X are in a pandas DataFrame and the target y is a pandas Series.

Requirements

  1. Split the data into train/test using stratification to preserve class ratios.
  2. Preprocess features:
    • Numeric: impute missing values and standardize.
    • Categorical: impute missing values and encode safely.
  3. Resample only the training data (avoid leakage) using a suitable method:
    • If only numeric features: SMOTE is acceptable.
    • If mixed types: use SMOTENC to correctly handle categorical features.
  4. Train a reasonable baseline classifier (e.g., logistic regression or tree-based model).
  5. Report precision, recall, and F1-score on the test set (per-class and macro/weighted averages are acceptable).

Deliverables

  • Reproducible Python code using scikit-learn and imbalanced-learn that implements the above and prints metrics on the held-out test set.
  • Brief comments justifying major choices (resampling method, pipeline order).

Clarifying Questions to Ask Guidance

  • Clarify users, core use cases, read/write patterns, scale, latency, availability, and data retention.
  • State explicit assumptions before making sizing or architecture decisions.
  • Prioritize the functional path first, then address reliability, security, observability, and rollout.

What a Strong Answer Covers Guidance

  • A scoped requirements summary with concrete non-goals and success metrics.
  • ML-specific data, model, evaluation, serving, and monitoring choices.
  • Reasoned trade-offs among simple and scalable designs, including bottlenecks and failure modes.
  • A validation, monitoring, migration, and launch plan appropriate for the risk level.

Follow-up Questions Guidance

  • What breaks first at 10x traffic or data volume?
  • How would you degrade gracefully during dependency failures?
  • What metrics and alerts would prove the design is healthy after launch?

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More ML System Design•More DRW•More Machine Learning Engineer•DRW Machine Learning Engineer•DRW ML System Design•Machine Learning Engineer ML System Design