How to decide if users need a new feature
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You are a Data Scientist at a social app. The product team proposes a new in-app feature (e.g., a new sharing surface). You have event-level data and can run experiments.
Answer the following:
1) **Using the data you already have (observational / pre-launch data), how would you identify whether users need this feature?**
- Be explicit about what “need” means and what signals would support it.
- Include how you would segment users and avoid misleading conclusions.
2) **Assume you can obtain any data you want. What additional data would you collect to answer (1) better?**
- Include instrumentation you would add and any qualitative inputs you’d combine with quantitative data.
3) **How would you measure whether the feature is successful after launch?**
- Define a North Star metric and 2–4 supporting/guardrail metrics.
- Describe an A/B test design (randomization unit, duration, power/MDE considerations).
4) **How would you clarify the feature goal and trade-offs with stakeholders?**
- Give examples of common trade-offs (e.g., engagement vs. quality, retention vs. spam/abuse, creator vs. consumer outcomes).
5) **How would you detect and handle novelty effects?**
- Describe what you would look for in time series and what analysis or experimental design changes you would use to ensure the result is durable.
Assumptions: users can generate multiple events per day; you can track impressions, clicks, downstream actions, and retention. State any additional assumptions you need.
Quick Answer: This prompt evaluates experimentation design, causal inference, product analytics, and instrumentation skills; it falls under Analytics & Experimentation and targets applied product-level data science at a strategic/analytical abstraction level, commonly asked to assess judgment on metric choice, segmentation, and trade-off reasoning.