{"blocks": [{"key": "b347c4f7", "text": "Scenario", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "369c15d2", "text": "Product team wants metrics and experiment design to reduce spam without harming normal user experience.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "86d5253f", "text": "Question", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "b2dab238", "text": "Without a classifier table, what alternative metrics would you track to monitor spam and overall user experience? If report rate declines, what potential causes could explain it and what extra metrics would you examine? When running an anti-spam A/B test where spammers are rare, how would you select test and control groups?", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "d24c119a", "text": "Hints", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "cd53eaa5", "text": "Consider message volume, unique senders, report-per-message, acceptance rate, stratified sampling, power.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}], "entityMap": {}}