Design a Comment-Likelihood Prediction System with a Focus on Feature and Model Serving

Design the machine-learning system behind comment likelihood on a large discussion platform where posts belong to communities: given a user and a post, estimate the probability that the user will comment on that post. Cover the problem framing, training data, features and model, but expect most of the discussion to be about **feature serving** and **model serving**: how every feature the model needs is computed, stored and fetched at request time, and how the model is deployed and scored within the request's latency budget. ```hint Sort features by freshness Some signals change once a day and others change every few seconds while a post is taking off. Decide how fresh each feature must be; the pipeline and the store that produce it follow from that. ``` ```hint Score the whole candidate list at once A single request asks for scores on many posts for the same user. Look for work that can be done once per request instead of once per post. ``` ### Clarifying Questions - Where is the score consumed: ranking a personalized feed, ranking posts inside one community, or choosing which posts to send in notifications? This changes the number of candidates and the latency budget. - What counts as a positive: a top-level comment only, or any reply in the post's thread, and within how long after the post was shown? - What latency budget does the ranking stage have, and how many candidates does it score per request? - Which infrastructure already exists: an event stream, a feature store, a model-serving platform? ### What a Strong Answer Covers - A precise label (event, time window, impression source), with position bias and class imbalance handled and outputs kept calibrated. - Features grouped by entity and freshness, with batch, streaming and request-time pipelines feeding a low-latency online store and a point-in-time-correct offline store. - Training-serving consistency: shared feature definitions or logged serving features, and how skew is detected. - A model-serving path with a latency budget, batched candidate scoring, versioned deployment of the model together with its feature schema, safe rollout, timeouts and fallbacks. - Monitoring of feature freshness, prediction calibration and online metrics, including guardrails against rewarding low-quality engagement. ### Follow-up Questions - How would you prevent training-serving skew when a feature's streaming computation differs slightly from its batch backfill? - A trending post's comment count changes many times per second and is read by many concurrent requests. How do you serve it fresh without overloading the online store? - How would you roll out a new model version safely, and how would you detect a calibration regression after launch? - How do you score a brand-new user, or a post that was created a minute ago?