Defining Success Metrics for Intercom's New SMS Channel
Company: Intercom
Role: Product Manager
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Other
Intercom has rolled out SMS as a new way to engage users through text messages. How would you decide whether this feature is a success?
```hint Success for whom
More than one party has to get something out of a text before it counts as a win. Work out what each of them wants from SMS, then choose the measure that best reflects all of them at once.
```
### Clarifying Questions
- Is SMS a channel that businesses on Intercom use to text their own customers, or a way for Intercom to reach the people who use Intercom itself?
- Does the feature cover two-way support conversations, outbound messages such as reminders and campaigns, or both?
- Is SMS billed separately, for example per message, so that revenue and margin are part of success, or is it bundled into existing plans?
- How long has the feature been live: are we judging early adoption or its long-term effect on the business?
### What a Strong Answer Covers
- The goals SMS serves for the businesses sending messages, the people receiving them, and Intercom itself
- One primary success metric with a precise definition and a reason it beats counting messages sent
- Supporting metrics across setup, adoption, recipient engagement and business outcomes
- Guardrails specific to texting: opt-outs, spam complaints, delivery failures, consent compliance and per-message cost
- How to separate the feature's causal effect from the fact that healthier accounts adopt new features first
### Follow-up Questions
- Reply rates on SMS are high, but opt-out rates keep climbing month over month. What do you conclude, and what would you change?
- Most new SMS volume replaces emails that were already performing well. Is the feature still a success?
- How would you show that adopting SMS improves customer retention rather than just correlating with it?
Overview: A product analytics question about how to judge whether Intercom's new SMS engagement feature is succeeding. It tests defining goals for the businesses sending texts, their recipients and Intercom, choosing a primary metric with texting-specific guardrails, and separating causal impact from adoption bias.