Calling Feature Launch: Success Metrics, Retention, Guardrails, and Decision Rubric
Context: You are launching a new feature in a consumer calling product that also supports messaging. Design the metrics framework for the launch.
Tasks:
-
Critique "time spent" as a primary success metric for a calling feature.
-
Propose a better primary metric set with precise, implementation-ready definitions and guardrails.
-
Define retention properly using cohorts. Provide explicit formulas for:
-
Day-1 retention
-
Week-1 retention
-
Rolling 28-day retention
Use the activity definition active_if = placed_or_received_call OR sent_message. Specify the numerator, denominator, cohorting date, and whether resurrected users are included.
-
Outline how you would monitor for negative side effects (e.g., call failure rate, complaint rate) and how to combine metrics into a decision rubric for launch/rollback/iterate decisions.