How to evaluate similar-listing notifications feature

Case study (Marketplace product analytics)

Context: Circle is a US marketplace app for buying and selling second‑hand products. On a product listing page, a buyer can click “send message” to contact the seller. Each message sent counts as one listing interaction.

The team is considering (and then ships) a new feature on product listings:

• Buyers can opt into reminders/notifications such as “similar listings you may like.”
• When similar products become available, the buyer receives a notification.

Part A — Should we build it?

How would you decide whether this is a good idea for the product? In your answer, cover:

• The user problem and hypothesis
• What data you would analyze before building (opportunity sizing)
• What success would look like and what could go wrong
• What MVP / rollout plan you would propose if you were uncertain

Part B — It’s implemented. How do we measure impact?

The developers have shipped the functionality. How would you understand its impact and determine whether it is a successful feature?

Be specific about:

• Primary success metric(s) vs diagnostic metrics vs guardrail metrics
• Experiment or quasi-experiment design (unit of randomization, control, duration)
• Key pitfalls (selection bias from opt-in, notification fatigue, interference/network effects, seasonality)
• How you would interpret results and decide to iterate, roll out, or roll back