Test if social users are more engaged
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: Hard
Interview Round: Onsite
A PM has a hypothesis: **users who use “social” apps are more engaged on a regular basis than users who use “game” apps.**
You have the same tables:
- `user_activity(user_id, date, app_id, session_id, duration)`
- `apps(app_id, app_name, app_category)`
## Task
Describe how you would **evaluate** this hypothesis using historical data.
Your answer should cover:
1) How you would define **engagement** (choose a primary metric and 1–2 diagnostic metrics).
2) How you would define comparison groups when many users use **both** categories (e.g., “social-only”, “game-only”, and mixed users; or thresholding by % time).
3) What statistical test/model you would use to compare groups, and what assumptions you’d check.
4) What you would do if the “social-only” and “game-only” groups are small (e.g., n≈300 each) but the mixed group is very large (e.g., n≈100,000).
5) Key confounders/biases you would worry about and how you’d mitigate them (e.g., user tenure, overall activity level, seasonality).
Quick Answer: This question evaluates a data scientist's competencies in observational analytics, engagement metric selection, cohort construction for overlapping behaviors, statistical comparison and bias/confounding assessment, and sits in the Analytics & Experimentation domain of Data Science.