Design A/B Tests for Banner Ad and Group-Story Feature
Company: Snapchat
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
##### Scenario
Product decision cases:
1) whether to add a banner ad,
2) whether to launch a group-story feature.
##### Question
What key metrics would you track to measure the impact of adding a banner ad versus not adding it? How would you design and analyze an A/B test to decide if the banner should be launched, including handling accidental clicks? For the group-story feature, which success metrics would you define and how would you set up the experiment (units, sample size, duration) to decide on launch?
##### Hints
Think CTR, dwell time, efficiency, retention, experiment design, guardrails, segmentation.
Quick Answer: This question evaluates a data scientist's skills in experimental design, metrics selection, causal inference, and interpretation of engagement and monetization signals for product changes like banner ads and group-story features.