Design a leak-free time-split model
Company: Stripe
Role: Data Scientist
Category: Machine Learning
Difficulty: hard
Interview Round: Technical Screen
You have one week (recommended 4–6 hours of effort) to deliver slides plus runnable code that predicts each active user's probability of making a purchase in the next 30 days as of a snapshot time. Requirements: (1) Define snapshot_ts = 2025-09-01 00:00:00 UTC; predict whether an order occurs in [snapshot_ts, snapshot_ts + 30d). (2) Specify exactly how you will prevent label leakage, including late-arriving events (events whose arrival_time > event_time) and any features computed after snapshot_ts. (3) Propose a minimal but strong baseline and a main model, justify choosing between logistic regression with monotonic/regularized features versus gradient-boosted trees; list the top 5 engineered features you would start with and why. (4) Choose the primary optimization metric under class imbalance (e.g., PR-AUC vs ROC-AUC) and the business metric you will report on slides; explain trade-offs. (5) Describe a temporal cross-validation scheme (rolling-origin/blocked) that produces an honest estimate; include how you will tune hyperparameters quickly without overfitting given the time budget. (6) Explain how you will detect and mitigate data/label leakage, target leakage, and train-test contamination; include at least two concrete checks you would code. (7) Outline your calibration plan (e.g., Platt scaling vs isotonic), threshold selection for an email targeting use-case with a cost per send, and how you'll communicate calibration and expected impact in slides. (8) Include a minimal ablation plan you would run within the timebox (which features/modeling choices you would drop first if time runs short), and the exact contents of 5–7 slide headlines that tell a compelling story to a hiring manager.
Overview: This question evaluates a Data Scientist's competency in time-aware predictive modeling, including label leakage prevention for late-arriving events, temporal cross-validation, feature engineering, model selection (logistic regression vs gradient-boosted trees), calibration and thresholding for costed actions.
Read the full Stripe Data Scientist interview experience this question came from