{"blocks": [{"key": "c2f8048b", "text": "Scenario", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "efdac42c", "text": "As part of the Ads team you have trained a new ad-recommendation model and need to decide if it should replace the current model.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "1ad6ca2e", "text": "Question", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "d96a3cce", "text": "Design an experiment to evaluate the new recommendation model against the incumbent.", "type": "unordered-list-item", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "9f04ca0c", "text": "What primary and guardrail metrics would you track? How would these differ when presenting results to the CFO versus the CGO (growth)?", "type": "unordered-list-item", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "c25d9680", "text": "State your final recommendation framework for launch or rollback.", "type": "unordered-list-item", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "49b417dc", "text": "Hints", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "e3a7f83b", "text": "Cover experiment design (A/B, traffic split, duration), metrics (revenue, CTR, ROI, user retention), statistical significance and trade-offs for finance vs growth.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}], "entityMap": {}}