Define Metrics and Account for Network and Novelty Effects

Q: Define Metrics and Account for Network and Novelty Effects

This question evaluates a data scientist's ability to define a primary goal metric plus driver and guard-rail metrics for an in-app feedback measurement system while explicitly accounting for network interference and temporal novelty effects on outcomes.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Evaluating Notification-Triggered In-App Surveys

Scenario

Meta’s notification system triggers optional in-app surveys to measure user sentiment after notifications are sent.

Task

Define:
1. A primary goal metric.
2. At least one driver metric.
3. At least one guard-rail metric.
Explain how you would explicitly account for potential network effects and novelty effects when interpreting these metrics.

Hints

Clarify the business objective, user experience risks, and externalities.
Propose a measurement technique for network effects and novelty effects.

Define Metrics and Account for Network and Novelty Effects

Evaluating Notification-Triggered In-App Surveys

Scenario

Task

Hints

Solution

Comments (0)