Classify Imbalanced Tabular Data with Missing Values, Then Scale to Billions of Rows

In this ML coding round you are given a tabular dataset for a binary classification task. About 20% of the labels are positive, and some feature values are missing. Write Python code that preprocesses the data and trains classifiers; the reported approach trained a logistic regression and a random forest and compared them. Searching the web for scikit-learn function names and syntax was allowed. After the coding, the interviewer asked about evaluation metrics, feature engineering for decision trees, and what changes at billions of records. The dataset's columns were not reported. Assume a pandas DataFrame `df` with a binary `label` column and a mix of numeric and categorical feature columns, with missing values in both kinds. ### Constraints and Clarifications - Not reported: the dataset size, what a row represents, the time limit, and which metric the interviewer cared about most. Ask, or state your assumptions. - The code should run end to end. Syntax lookups are allowed, so the time should go into decisions rather than remembered signatures. ### Clarifying Questions - What does a row represent, and is there a timestamp or a repeated entity id that the train/test split must respect? - Which error costs more, a missed positive or a false alarm? Will the output be used as a ranking, a yes/no decision, or a probability? - Are values missing at random, or might the fact that a value is missing say something about the label? - How many rows are there, and do any categorical columns have very many distinct values? ### Part 1 — Preprocess and train Load the data, handle the missing values and the categorical columns, split the data, and train a logistic regression and a random forest. Report how each model performs on held-out data. ```hint Fit preprocessing on training rows only Decide where imputation and encoding are fitted so that no statistic from held-out rows leaks into training, and whether the fact that a value was missing should survive imputation. ``` #### What This Part Should Cover - A leak-free preprocessing pipeline: imputation, encoding, and scaling only where the model needs it - A stratified split or cross-validation that keeps the 20% positive rate in every fold - Handling the imbalance with class weights, resampling inside training folds only, or threshold choice - A like-for-like comparison of the two models ### Part 2 — Metrics for the imbalanced label Which evaluation metrics would you use for this imbalanced problem, and why? ```hint What does a useless model score Work out what accuracy and each candidate metric would be for a model that always predicts the majority class, or that scores rows at random. ``` #### What This Part Should Cover - Why accuracy misleads at this positive rate - Threshold-free ranking metrics and their baselines - Threshold-dependent metrics tied to the cost of each kind of error, and how the threshold is chosen - Probability quality when the scores are used as probabilities ### Part 3 — Feature engineering for decision trees How would you do feature engineering for a decision-tree model, and how does it differ from what the logistic regression needed? ```hint What can a split not express A tree splits on one feature at a time against a threshold. Think about which transformations change nothing for it, and which relationships it struggles to represent. ``` #### What This Part Should Cover - Transformations that trees do not need - Encoding categorical features for trees, including high-cardinality ones - Representing missing values, and building features that axis-aligned splits cannot capture easily - Avoiding leakage when building target-derived or aggregate features ### Part 4 — Billions of records The dataset grows to billions of records. Which model would you choose, and how would you engineer the categorical features? ```hint Memory, passes and cardinality Consider which models can be trained in a streaming or distributed way, whether every record is needed, and what happens to one-hot encoding when a column has millions of categories. ``` #### What This Part Should Cover - A model choice justified by training cost, serving cost and accuracy at this scale - Sampling with its correction, and distributed or out-of-core training - Scalable categorical encodings and their trade-offs - Validation at scale, such as a time-based holdout ### What a Strong Answer Covers - Clarifies the data, the cost of errors and how predictions are used before coding - Working, leak-free code that runs end to end on the given data - Treats the 20% positive rate correctly in training, in metric choice and in thresholding - Explains model-specific feature engineering instead of applying one recipe to every model - Makes scale-driven choices with explicit trade-offs ### Follow-up Questions - In production the positive rate drops from 20% to 5%. Which of your metrics and thresholds change, and what would you recalibrate? - How would you test whether missingness itself predicts the label, and what would you do if it does? - Your random forest wins on PR AUC, but its predicted probabilities are poorly calibrated. Does that matter, and how would you fix it? - A product partner asks why one particular record was scored as positive by the random forest. How do you explain it?