PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/Snapchat

Design an experiment for spam filtering impact

Last updated: Mar 29, 2026

Quick Overview

This question evaluates experimental design and causal inference skills for measuring the effect of a stricter spam filter on same-day friend-request acceptance, covering hypothesis specification, interference and randomization choices, metric definition and windows, power calculation, and handling incomplete labels.

  • hard
  • Snapchat
  • Analytics & Experimentation
  • Data Scientist

Design an experiment for spam filtering impact

Company: Snapchat

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

You plan to launch a stricter spam filter and want to understand its impact on friendship dynamics and same-day acceptance. a) State clear hypotheses: primary (effect on same-day acceptance rate) and at least two guardrail metrics (e.g., total requests sent, approval latency, false-positive spam rate). Specify null/alternative precisely. b) Propose the experimental unit and randomization scheme that mitigates network interference (e.g., cluster by requester vs. recipient). Justify your choice and discuss spillover risks. c) Define primary and secondary metrics, including exact measurement windows and UTC date boundaries. Explain how to handle delayed approvals that cross days. d) Outline your sample-size/power plan: assumed baseline same-day acceptance, minimum detectable effect, variance source, test duration, and a plan for sequential looks while controlling Type I error. e) Describe how incomplete spam labels in users could bias your metrics and two mitigation strategies (e.g., unknown bucket + sensitivity bounds; propensity/inverse-probability weighting if labeling is MAR). Explain how you would report results with these adjustments.

Quick Answer: This question evaluates experimental design and causal inference skills for measuring the effect of a stricter spam filter on same-day friend-request acceptance, covering hypothesis specification, interference and randomization choices, metric definition and windows, power calculation, and handling incomplete labels.

Related Interview Questions

  • Decide whether to launch Group Story - Snapchat (Medium)
  • Design and analyze a banner A/B test - Snapchat (hard)
  • Design A/B Test for New Recommendation Algorithm Launch - Snapchat (medium)
  • Design A/B Tests for Banner Ad and Group-Story Feature - Snapchat (medium)
  • Determine Optimal Energy Project for 10% ROI Target - Snapchat (medium)
Snapchat logo
Snapchat
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
2
0

Experiment Design: Stricter Spam Filter Impact on Friend Requests

Context

You run a social app with a friend-request system. A stricter spam filter will score and potentially block outgoing requests before delivery to recipients. You want to measure its impact on same-day acceptance behavior while protecting sender and recipient experience.

Assumptions for clarity (adjust if needed):

  • A "request" is created at send_time and is either delivered (passes filter) or blocked (filtered as spam).
  • A delivered request can be accepted at any later time.
  • Dates and windows use UTC boundaries.

Tasks

a) Hypotheses

  • Define a primary hypothesis on the same-day acceptance rate.
  • Define at least two guardrail hypotheses (e.g., total requests sent/delivered, approval latency, false-positive spam rate).
  • State the null and alternative precisely.

b) Experimental unit and randomization scheme

  • Propose the experimental unit and a randomization approach that mitigates network interference (cluster by requester vs. recipient, etc.).
  • Justify the choice and discuss spillover risks.

c) Metrics and windows

  • Define primary and secondary metrics with exact measurement windows and UTC date boundaries.
  • Explain how to handle approvals that occur on days after the request is sent.

d) Sample size and power plan

  • Provide assumed baseline same-day acceptance, MDE, variance source, test duration.
  • Describe sequential looks and Type I error control.

e) Incomplete spam labels

  • Explain how incomplete/unknown ground-truth labels could bias metrics.
  • Propose two mitigation strategies (e.g., unknown bucket + sensitivity bounds; propensity/inverse-probability weighting if MAR) and how you would report adjusted results.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Snapchat•More Data Scientist•Snapchat Data Scientist•Snapchat Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.