Build Premium User Propensity Model
Quick Overview
This question evaluates an engineer's competency in end-to-end machine learning system design for user conversion propensity, covering problem framing, labeling strategy without leakage, feature engineering across behavioral, content, social and device signals, embedding usage, model families and loss functions, class imbalance handling, and offline/online evaluation, calibration and deployment considerations. Commonly asked in Machine Learning interviews, it assesses the ability to translate business objectives and prediction horizons into practical modeling and operational choices, testing both conceptual understanding and practical application across data modeling, evaluation, calibration, thresholding, and downstream product integration.
Design an end-to-end modeling approach to identify free-tier users who are likely to convert to a premium subscription.
Discuss:
- the business objective and prediction horizon,
- how to define positive and negative labels without leakage,
- what data you would collect,
- feature engineering from behavioral, content, social, and device signals,
- when embeddings would be useful and how to generate them,
- candidate model families,
- appropriate loss functions,
- handling class imbalance,
- offline and online evaluation,
- calibration, thresholding, and how predictions would be used by downstream product teams.