Predict Customer Churn with Machine Learning Workflow
Company: TikTok
Role: Data Scientist
Category: Machine Learning
Difficulty: medium
Interview Round: Onsite
##### Scenario
A subscription platform wants to predict whether a customer will churn in the next month.
##### Question
Outline the end-to-end workflow—from feature engineering through model deployment—to build a churn predictor. 2. Which evaluation metrics would you prioritize and why? 3. How would you handle severe class imbalance during training?
##### Hints
Talk about train/validation split, cross-validation, ROC-AUC, precision-recall, SMOTE/weighted loss, monitoring.
Quick Answer: TikTok data scientist machine learning prompt on monthly churn prediction, covering label construction, feature engineering, time-aware validation, class imbalance, metrics, deployment, monitoring, and retention impact.
Predict Customer Churn with Machine Learning Workflow
TikTok
Jul 12, 2025, 6:59 PM
mediumData ScientistOnsiteMachine Learning
20
0
Predict Monthly Customer Churn With an End-to-End ML Workflow
A subscription platform wants to predict whether a customer will churn in the next month.
Assume churn means a subscriber is no longer active by the end of the next 30 days. Features used at prediction time must only use information available up to the scoring date.
Constraints & Assumptions
Cover the full workflow from problem framing to deployment and monitoring.
Use time-aware splits to prevent leakage.
Discuss severe class imbalance.
Prioritize metrics based on business decisions, not only generic model accuracy.
Clarifying Questions to Ask Guidance
What exact churn definition should we use: cancellation, failed renewal, inactivity, downgrade, or no payment?
What intervention will the model trigger, and what are the costs of false positives and false negatives?
How often will the model score users?
Are labels delayed or censored?
What a Strong Answer Covers Guidance
Problem framing: prediction horizon, label construction, scoring cadence, and leakage-safe reference dates.
Data sources: subscription/billing, usage, engagement, support, payments, marketing, device, geography, and plan attributes.
Feature engineering: recency/frequency, tenure, payment failures, usage trends, content affinity, support friction, price/discount status, and rolling windows.
Train/validation/test strategy using time-based splits, cross-validation where appropriate, and feature-store consistency.
Model choices such as logistic regression, gradient boosting, random forest, survival models, or calibrated classifiers.
Metrics: PR-AUC, recall at budget, precision, lift/gains, calibration, ROC-AUC, business profit/cost curves, and segment performance.
Class imbalance handling: class weights, threshold tuning, resampling, focal loss, anomaly framing where appropriate, and leakage-safe SMOTE or oversampling.
Deployment, monitoring, drift detection, recalibration, retraining triggers, and intervention measurement.
Follow-up Questions Guidance
Why can accuracy be misleading for churn?
How would you pick the threshold for retention offers?
How would you measure whether the churn model improves retention?
What features are likely to leak future information?