Evaluate Recommendation Feature with Historical Data Analysis
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
##### Scenario
The company is considering launching new recommendation-system features and wants to judge their value before any live deployment.
##### Question
Using only historical data, how would you evaluate whether releasing this recommendation feature is a good or bad idea? Detail the analyses, metrics, and assumptions you would use (do not answer "run an A/B test").
##### Hints
Think offline replay, counterfactual evaluation, uplift or propensity modeling, simulation, historical hold-out metrics.
Quick Answer: This question evaluates a data scientist's competence in offline recommendation evaluation, covering skills in causal inference, counterfactual estimation, metric selection, assumption articulation, and validation using historical logs.