Ads platform wants to validate engineers’ claim that a new ML ranking model outperforms the existing recommender.

How would you design an experiment to evaluate the new ML recommendation system? Which primary and guard-rail metrics would you monitor and why? An A/B test shows a 5% lift in CTR—how do you judge practical significance? CTR doubles for Indian males aged 18-55—what might this indicate and what next steps would you take? If the test yields +5% CTR and +5% revenue, would you roll the model out globally? Explain your decision process.

Discuss randomization, sample size, heterogeneous effects, business trade-offs, and ethical checks.

Meta Data Scientist interview question: Design an Experiment to Evaluate New ML Model. {"blocks": [{"key": "54f3ac49", "text": "Scenario", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {...