PracHub
QuestionsPremiumCoachesLearningGuidesInterview Prep
|Home/Analytics & Experimentation/Meta

Define success metrics beyond time spent

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's competency in product analytics and experimentation, focusing on metrics design, cohort-based retention measurement, identification of negative side effects, and construction of decision rubrics for feature launches.

  • hard
  • Meta
  • Analytics & Experimentation
  • Data Scientist

Define success metrics beyond time spent

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Onsite

For a calling product, define success metrics for a feature launch. Critique "time spent" as a primary metric and propose a better primary metric set (with precise definitions) and guardrails. Then define retention properly: give cohort-based formulas for day-1, week-1, and rolling 28-day retention (active_if = placed_or_received_call OR sent_message). Specify numerator/denominator, cohorting date, and whether to include resurrected users. Finally, outline how you would monitor for negative side effects (e.g., call failure rate, complaint rate) and how you'd combine metrics into a decision rubric.

Quick Answer: This question evaluates a data scientist's competency in product analytics and experimentation, focusing on metrics design, cohort-based retention measurement, identification of negative side effects, and construction of decision rubrics for feature launches.

Related Interview Questions

  • Measure scheduled posts feature success - Meta (medium)
  • Estimate ads ranking revenue impact - Meta (medium)
  • How should you evaluate unconnected content? - Meta (medium)
  • Should WhatsApp launch group calls? - Meta (medium)
  • How would you grow Meta products? - Meta (medium)
Meta logo
Meta
Oct 13, 2025, 9:49 PM
Data Scientist
Onsite
Analytics & Experimentation
1
0
Loading...

Calling Feature Launch: Success Metrics, Retention, Guardrails, and Decision Rubric

Context: You are launching a new feature in a consumer calling product that also supports messaging. Design the metrics framework for the launch.

Tasks:

  1. Critique "time spent" as a primary success metric for a calling feature.
  2. Propose a better primary metric set with precise, implementation-ready definitions and guardrails.
  3. Define retention properly using cohorts. Provide explicit formulas for:
    • Day-1 retention
    • Week-1 retention
    • Rolling 28-day retention Use the activity definition active_if = placed_or_received_call OR sent_message. Specify the numerator, denominator, cohorting date, and whether resurrected users are included.
  4. Outline how you would monitor for negative side effects (e.g., call failure rate, complaint rate) and how to combine metrics into a decision rubric for launch/rollback/iterate decisions.

Solution

Show

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Meta•More Data Scientist•Meta Data Scientist•Meta Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 8,000+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.