Design Metrics to Track and Analyze Spam Impact

##### Scenario Product team wants metrics and experiment design to reduce spam without harming normal user experience. ##### Question Without a classifier table, what alternative metrics would you track to monitor spam and overall user experience? If report rate declines, what potential causes could explain it and what extra metrics would you examine? When running an anti-spam A/B test where spammers are rare, how would you select test and control groups? ##### Hints Consider message volume, unique senders, report-per-message, acceptance rate, stratified sampling, power.