Measure a friend-recommendation launch

Q: Measure a friend-recommendation launch

This question evaluates experiment design, metric selection, causal inference and business-metric translation for social features, including defining primary and guardrail metrics, choosing experimental units to mitigate network interference, and estimating revenue impact from observed lifts.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

A new friend-recommendation algorithm ships behind a feature flag. Design how you will measure success and decide whether to launch:

State no more than 3 primary metrics and 3 counter/guardrail metrics, with precise 14-day attribution windows (e.g., confirmed-friends-per-DAU, acceptance rate, qualified exposures-per-DAU; guardrails: spam reports, hide-rate, session length).
Define the experimental unit and randomization layers (user-level, network clusters for interference) and how you’ll handle network effects (ghost/long-term holdouts or staggered rollouts).
Suppose in the first 14 days: MAU +5% (p<0.05), engagement per user −3% (p<0.05), confirmed friends per DAU +2% (p=0.07). Make a launch/no-launch recommendation and justify with a quantitative decision rule.
Translate a +2% lift in confirmed-friends-per-DAU into an estimated annual revenue/profit delta given assumed ARPU and margin; specify the minimum economically-meaningful effect that warrants a full rollout.
Describe how you will monitor post-launch for regression and set automatic rollback criteria.

Measure a friend-recommendation launch

Overview

Comments (0)