Modify Instagram Feature: Track User Engagement Metric
Company: Yelp
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
##### Scenario
A social-media company is considering modifying a disliked Instagram-like feature and must run an A/B test before shipping the change.
##### Question
Pick a popular consumer app (e.g., Instagram) and name a feature you do not like. What single or composite metric would you track to measure user response? How would you estimate the required sample size and decide how long the test should run? State your assumptions and calculations. Which user segment or geography would you target first and why? What statistical test would you apply to compare control and treatment groups and why is it appropriate? Suppose the overall test is not statistically significant but the treatment group shows higher engagement. How would you interpret this outcome? Given the above, would you
(a) fully launch the change,
(b) rerun or extend the experiment, or
(c) dig into a specific subgroup such as younger users? Justify your choice.
##### Hints
Discuss metric choice, power analysis, duration, segmentation, test selection, and post-test decisions.
Quick Answer: This question evaluates a data scientist's skills in experimental design, metric definition, statistical power and sample-size estimation, hypothesis testing, and segmentation interpretation for product experiments.