Predict Click Probability for Music Recommendations

Design a machine-learning system that predicts the probability that a user will click a recommended song. Explain how you would represent songs, handle extremely sparse features, choose a prediction algorithm, and evaluate the resulting click-through-rate predictions. ### Requirements and Constraints For this exercise, assume the system receives a user, a candidate song, and recommendation context, then outputs a click probability for that impression. Candidate retrieval is outside the initial scope. Training data contains logged impressions and whether each impression produced a click within a defined attribution window. These are explicit task assumptions; the attribution window, serving latency target, and available song metadata must be clarified. ### Clarifying Questions - What counts as a click, and how are repeated impressions or delayed clicks attributed? - Which user history, song metadata, interaction data, or audio-derived features are available at prediction time? - Are sparse features high-cardinality identifiers, infrequently observed categories, or mostly missing measurements? - Must the probability be calibrated, used only for ranking candidates, or both? ```hint Distinguish an unseen song from an unobserved feature A song with little interaction history may still have usable metadata or audio information. Decide how its representation is formed before it has enough clicks for a reliable learned identifier embedding. ``` ### What a Strong Answer Covers - An impression-level label and feature construction that avoids using information unavailable when the recommendation was shown. - Song representations and fallbacks for new or rarely observed songs. - A specific treatment of sparse categorical features, rare categories, missingness, and the size of the feature space. - A comparison between a simple probabilistic baseline and a model capable of useful nonlinear or cross-feature interactions. - Offline evaluation of probability quality and ranking behavior with realistic time-based or user-aware validation. - Online evaluation and secondary listening-quality measures that prevent optimizing clicks alone. ### Follow-up Questions 1. How would you score a newly released song with useful metadata but no historical interactions? 2. What could cause a model with better offline AUC to produce poorly calibrated click probabilities? 3. How would a change in which songs are displayed affect the training labels collected by the system?