Determine Key Metrics for Circle's Success Evaluation

Q: Determine Key Metrics for Circle's Success Evaluation

This question evaluates a data scientist's competency in defining north-star metrics and metric frameworks, designing retrospective analyses with bias and confounding mitigation, and normalizing time-series engagement signals for fair comparison.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Meta.

Q: What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Meta during technical interviews.

Question

Circle (Group-like) Feature: Metrics and Retrospective Sizing Analysis

Context

You are evaluating a new social feature called Circle (similar to Groups). Assume:

Circle is the only feature on the platform, so guardrails/cannibalization are out of scope.
You only have retrospective (observational) data; you cannot run an A/B test yet.
You’ve also been shown three time-series plots of total comments divided by total posts for three different products with different user bases.

Tasks

Define the north-star metric and a concise metric framework to measure Circle’s success.
Using only retrospective data, decide whether Circle should be optimized for small groups (≈5–6 members) or for large groups (tens to hundreds). Outline the analysis design, key variables, and how you will mitigate bias/confounding.
You are shown three lines (time series) of total comments per total posts for three different products with different user bases. Can these lines be directly compared? Why or why not, and how would you normalize them for a fair comparison?

Determine Key Metrics for Circle's Success Evaluation

Quick Overview

Circle (Group-like) Feature: Metrics and Retrospective Sizing Analysis

Context

Tasks

Solution

Comments (0)