{"blocks": [{"key": "6e250dbb", "text": "Scenario", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "503883f2", "text": "Trust & Safety data science: design metrics for a content-moderation A/B test and for evaluating a customer-service chatbot’s knowledge base.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "6c3b324f", "text": "Question", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "89171211", "text": "In a content-moderation A/B test where harmful-content prevalence is low, which short-term, user-centric metrics would you track to detect impact quickly and why? How would you design an experiment and select evaluation metrics to measure the quality and usefulness of a customer-service chatbot’s knowledge base?", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "17d2fcd1", "text": "Hints", "type": "header-two", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}, {"key": "353fa213", "text": "Consider immediate user actions: report rates, dismissals, session exits, latency; for chatbot, precision/recall of answers, deflection rate, CSAT; discuss experiment design and trade-offs.", "type": "unstyled", "depth": 0, "inlineStyleRanges": [], "entityRanges": [], "data": {}}], "entityMap": {}}