Design Metrics to Track and Analyze Spam Impact

Q: Design Metrics to Track and Analyze Spam Impact

This question evaluates a data scientist's ability to define proxy metrics, interpret noisy behavioral signals, and design statistically powered experiments for rare events, testing skills in product analytics, causal inference, A/B testing, and metric instrumentation.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Scenario

A messaging product team wants to reduce spam without harming normal user experience. You do not have access to a ground-truth spam classifier table.

Tasks

Metrics without a classifier: What proxy metrics would you track to monitor spam prevalence and overall user experience?
Interpreting a decline: If the user report rate (reports per message) declines, what plausible causes could explain it, and what additional metrics would you examine to disambiguate?
Experiment design with rare spammers: When running an anti-spam A/B test where spammers are rare, how would you select test and control groups to ensure power and minimize interference?

Hint: Consider message volume, unique senders, reports per message, acceptance rate of message requests, stratified sampling, and power.

Design Metrics to Track and Analyze Spam Impact

Scenario

Tasks

Solution

Comments (0)

Design Metrics to Track and Analyze Spam Impact

Overview

Scenario

Tasks

Solution

Comments (0)