##### Scenario Deciding whether to launch and how to evaluate a new Group Call feature for a communication app. ##### Question Using only the existing usage table, how would you decide if the product needs a Group Call feature? If additional resources were available, what extra data or research would you request and why? How would you set an upper limit on the number of participants in a group call? Define and justify a threshold. Design an A/B test for the Group Call feature. Detail hypothesis, metrics, experiment setup, sample-size and runtime calculations, guardrails, and potential pitfalls. Nine months after launch, what metrics and analyses would you use to measure the feature’s success? If there is no measurable impact on overall company metrics, is that good or bad? Should the feature be kept? Explain. Discuss trade-offs between optimizing ecosystem-level metrics versus focusing on users with poor call experiences. Identify likely post-launch drop-off points in the Group Call funnel and propose mitigations. ##### Hints Think metric definitions, leading vs. lagging indicators, experiment design best practices, and user-level diagnostic analyses.

This question evaluates a data scientist's competency in product analytics, experiment design, causal inference, instrumentation, metric definition, and user funnel diagnostics for a communications feature.

How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Meta.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Determine Group Call Feature Need and Evaluation Methods

Scenario

You are evaluating whether to build and launch a Group Call feature in a consumer communication app. You currently have access to a single “usage” table (assume it contains user-day level calling activity: user_id, date, total_call_minutes, calls_made, unique_contacts_called, and platform/device tier). No other data is guaranteed.

Tasks

Using only the existing usage table:
- How would you decide if the product needs a Group Call feature? Describe the analyses, metrics, and decision criteria you would use.
If additional resources were available:
- What extra data or research would you request and why? Be specific about how each addition would change your confidence or decision process.
Participant cap:
- How would you set an upper limit on the number of participants in a group call? Define a threshold and justify it.
A/B test design for Group Call:
- State your hypotheses (primary and secondary).
- Define success metrics and guardrails (quality, safety, cost).
- Describe the experiment unit, randomization, and setup to minimize spillovers.
- Provide sample size and runtime calculations with explicit assumptions and formulas.
- Call out potential pitfalls and how to mitigate them.
Nine months post-launch:
- What metrics and analyses would you use to assess long-term success? If there is no measurable impact on overall company metrics, is that good or bad? Should the feature be kept? Explain your criteria.
- Discuss trade-offs between optimizing ecosystem-level metrics versus improving outcomes for users with poor call experiences.
Funnel diagnostics:
- Identify likely post-launch drop-off points in the Group Call funnel and propose mitigations for each.

Scenario

Tasks

Using only the existing usage table:

How would you decide if the product needs a Group Call feature? Describe the analyses, metrics, and decision criteria you would use.

If additional resources were available:

What extra data or research would you request and why? Be specific about how each addition would change your confidence or decision process.

Participant cap:

How would you set an upper limit on the number of participants in a group call? Define a threshold and justify it.

A/B test design for Group Call:

State your hypotheses (primary and secondary).
Define success metrics and guardrails (quality, safety, cost).
Describe the experiment unit, randomization, and setup to minimize spillovers.
Provide sample size and runtime calculations with explicit assumptions and formulas.
Call out potential pitfalls and how to mitigate them.

Nine months post-launch:

What metrics and analyses would you use to assess long-term success? If there is no measurable impact on overall company metrics, is that good or bad? Should the feature be kept? Explain your criteria.
Discuss trade-offs between optimizing ecosystem-level metrics versus improving outcomes for users with poor call experiences.

Funnel diagnostics:

Identify likely post-launch drop-off points in the Group Call funnel and propose mitigations for each.

Determine Group Call Feature Need and Evaluation Methods

Quick Overview

Scenario

Tasks

Solution

Submit Your Answer to Earn 20XP

Determine Group Call Feature Need and Evaluation Methods

Quick Overview

Scenario

Tasks

Solution

Submit Your Answer to Earn 20XP