Design Experiment to Test New Hashtag Recommender Algorithm
Overview
This question evaluates a data scientist's experiment design competency, including A/B testing fundamentals, randomization unit choice, stratification/blocking, sample size and power considerations, guardrail metrics, and exposure/interference handling for online recommender systems.
Meta
Jul 12, 2025, 6:59 PM
Data Scientist
Onsite
Analytics & Experimentation
11
0
Experiment Design: Testing a New Hashtag Recommendation Algorithm
Context
A social app shows hashtag recommendations to users while composing posts. A new algorithm is proposed to increase hashtag adoption and downstream engagement without harming reliability or content quality.
Task
Design and run an online experiment to evaluate the new recommender. Focus on how you would choose test and control groups, and address:
- Randomization unit
- Stratification/blocking
- Sample size and statistical power
- Guardrail metrics
- Exposure overlap/interference
- Practical run plan (ramp, duration, analysis)
Solution
Login required
Show