End-to-End Tabular Modeling: Duplicate Keys, Imputation, One-Hot Encoding and Calibration

You are given a tabular dataset, much like a Kaggle competition dataset, and asked to build a model from scratch in a notebook. The model itself can be simple; a random forest is enough. The interview focuses on data handling and evaluation: exploratory analysis, data-quality checks and preprocessing first, then evaluating the fitted model and calibrating its predicted probabilities. The report names precision, recall and AUC as the metrics discussed, so assume a binary classification target; it does not describe the columns, so settle them with the interviewer. ### Clarifying Questions - What does one row represent, and which column or pair of columns is supposed to identify a row uniquely? - What is the target, how imbalanced is it, and is there a time column that the train and test split should respect? - Which error costs more here, a missed positive or a false alarm, so that a decision threshold can be chosen? - Will the predicted probabilities be used as probabilities (compared with a cost, or summed into an expected count), or only to rank rows? ### Part 1 — Explore the data and check its quality Load the data into a pandas DataFrame and run an initial exploratory pass. In particular, check whether the DataFrame has duplicates on a single key column and on a combination of two key columns, and say how you would handle what you find. Also profile missing values and outliers. ```hint Two kinds of duplicate Ask whether two rows sharing a key are identical copies or disagree in other columns, and whether either answer changes how you split the data for testing. ``` #### What This Part Should Cover - pandas checks for duplicates on one key and on a composite key, and how to inspect the offending rows - Telling exact duplicate rows from conflicting records that share a key, and the action for each - A missing-value profile per column and whether missingness relates to the target - Outlier detection on numeric columns, and deciding whether an extreme value is an error ### Part 2 — Preprocess: imputation, outliers and categorical encoding Build the preprocessing for numeric and categorical columns: impute missing values, treat outliers, and one-hot encode the categorical features, including categories that are missing or that appear only at prediction time. ```hint Where statistics come from Decide which rows each learned statistic (a median, the list of categories) is computed on, and what should happen when a category appears at prediction time that training never saw. ``` #### What This Part Should Cover - Numeric imputation and missing-value indicators - One-hot encoding with missing as its own category and unseen categories handled - Fitting every preprocessing step on training data only, inside a pipeline - Outlier treatment matched to the model family ### Part 3 — Train a baseline and evaluate it Fit a random forest and evaluate it. Define precision, recall and ROC AUC, explain what each one tells you about this model, and explain how you would choose the decision threshold. ```hint Thresholds Notice which of the three metrics depend on a threshold and which do not, and which data the threshold should be chosen on. ``` #### What This Part Should Cover - An evaluation split that accounts for duplicates and time ordering - Precision and recall from the confusion matrix at a chosen threshold, and their trade-off - ROC AUC as a threshold-free ranking metric, and when precision-recall AUC is more informative - Choosing the threshold from error costs on validation data ### Part 4 — Calibrate the probabilities The interviewer asks in detail about calibration: whether the model's predicted probabilities are calibrated, and how exactly you would calibrate them. Explain how to measure calibration and how to fix it. ```hint What a score should mean Ask what a predicted 0.8 should mean across many rows, and which rows a calibration map may be fitted on. ``` #### What This Part Should Cover - Measuring calibration: reliability diagram, Brier score or expected calibration error - Platt scaling versus isotonic regression, and when each fits - Fitting the calibrator on held-out predictions or by cross-validation, never on training predictions - The effect of calibration on AUC and on the decision threshold ### What a Strong Answer Covers - Spends the time on data checks before modelling, as the interview expects - Prevents leakage throughout: entities split across train and test, preprocessing fitted on training rows only, calibration on held-out rows - Builds one reproducible pipeline rather than ad hoc transformations - Justifies metric and threshold choices by the problem's error costs and class balance - Explains findings clearly while coding ### Follow-up Questions - The positive class is rare. Which metrics and which calibration method would you trust, and why? - You up-sampled positives to train the model. What happens to its predicted probabilities, and how do you correct them? - How would you check calibration for one subgroup, such as a single category, rather than overall? - Why can a random forest have a high AUC and still produce poorly calibrated probabilities?