
You are evaluating a change to the main social feed in a large-scale consumer app. Assume events are logged to a single analytics table with impression-level and action-level events.
Minimal events table (to ground definitions):
Specify:
(a) The primary success metric(s) with exact formulas (numerator/denominator and units).
(b) Guardrail metrics (e.g., latency, content quality, reliability) with formulas and units.
(c) The eligible population and unit of analysis (e.g., user-day, session, impression) for the primary metric and guardrails.
(d) Data hygiene rules: how to handle bots, empty sessions, outliers, and time zones.
(e) How each metric would be computed from the events table (outline the steps and any key joins/windowing).
Finally, discuss the trade-offs between engagement, quality, and reliability, and how your metric set balances these.
Login required