How to evaluate similar-listing notification feature
Company: Meta
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Technical Screen
You are a Data Scientist on a US C2C marketplace app (like Facebook Marketplace) where users buy/sell second-hand products.
**Current product behavior**
- Users browse product listings.
- If a buyer is interested in a listing, they can click **“Send message”** to contact the seller.
- Each message sent counts as **one listing interaction**.
**Proposed feature**
On a product listing, buyers can opt into **reminders/notifications** for **“similar listings you may like.”** If similar products become available, the buyer receives a notification.
Answer the following:
1) **Pre-launch / decision framing:** How would you decide whether this feature is a good idea for the product? Include:
- What user problem/hypothesis you are testing
- What success metrics you would expect to move (and why)
- Key tradeoffs and risks (e.g., notification fatigue)
- What data you would want to validate demand and estimate impact
2) **Post-implementation impact evaluation:** Assume engineers have shipped the functionality (or it can be enabled for some users). How would you measure its impact and determine whether it is successful? Include:
- Recommended experiment or causal design, unit of randomization, and key comparisons
- Primary metrics, secondary/diagnostic metrics, and guardrails
- How you would interpret outcomes, handle confounding/selection effects, and decide to launch/rollback
Quick Answer: Evaluates experiment design and causal inference skills—including hypothesis framing, metric selection, tradeoff analysis (e.g., notification fatigue), and data-driven product decision making—in the Analytics & Experimentation domain at a mid-to-senior product/data scientist abstraction level.