Design A/B Test to Evaluate New Video-Feed Feature
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
##### Scenario
A social media company is launching a short-video feed similar to TikTok and wants to evaluate a newly added feature affecting how users engage with the feed.
##### Question
Design an A/B test to measure the impact of the new video-feed feature. Cover experiment unit, randomization, success metrics, sample-size needs, guardrails and how you would monitor and conclude the test. Which primary and secondary metrics would you track and why? How would you handle possible novelty and seasonal effects?
##### Hints
Think about activation, retention, engagement and negative metrics; outline experiment design and decision criteria.
Quick Answer: This question evaluates a candidate's skills in experimental design, statistical inference, metric definition, and product analytics for measuring a new short-video feed feature, with attention to randomization, cross-device consistency, sample size, guardrail metrics, and interference risks.