Scenario

You are evaluating whether to build and launch a Group Call feature in a consumer communication app. You currently have access to a single “usage” table (assume it contains user-day level calling activity: user_id, date, total_call_minutes, calls_made, unique_contacts_called, and platform/device tier). No other data is guaranteed.

Tasks

1. Using only the existing usage table:
   • How would you decide if the product needs a Group Call feature? Describe the analyses, metrics, and decision criteria you would use.
2. If additional resources were available:
   • What extra data or research would you request and why? Be specific about how each addition would change your confidence or decision process.
3. Participant cap:
   • How would you set an upper limit on the number of participants in a group call? Define a threshold and justify it.
4. A/B test design for Group Call:
   • State your hypotheses (primary and secondary).
   • Define success metrics and guardrails (quality, safety, cost).
   • Describe the experiment unit, randomization, and setup to minimize spillovers.
   • Provide sample size and runtime calculations with explicit assumptions and formulas.
   • Call out potential pitfalls and how to mitigate them.
5. Nine months post-launch:
   • What metrics and analyses would you use to assess long-term success? If there is no measurable impact on overall company metrics, is that good or bad? Should the feature be kept? Explain your criteria.
   • Discuss trade-offs between optimizing ecosystem-level metrics versus improving outcomes for users with poor call experiences.
6. Funnel diagnostics:
   • Identify likely post-launch drop-off points in the Group Call funnel and propose mitigations for each.