Design success and guardrail metrics

You are launching a new recommendation module on a content platform. Design a metrics framework and an experiment plan. A) Define one primary success metric with an exact formula (include unit of randomization, numerator, denominator, inclusion/exclusion criteria, and a 28-day attribution rule). Define at least two guardrail metrics with clear thresholds and directions (e.g., bounce rate, crash rate, page latency) and explain trade-offs. B) Specify the randomization unit and bucketing strategy to avoid contamination across surfaces and sessions. Describe how you would detect and triage Sample Ratio Mismatch (SRM) and event-loss issues (e.g., via canary dashboards, heartbeat events), including a concrete SRM test and an acceptable p-value threshold. C) Explain how you will handle novelty effects and seasonality (e.g., ramp schedule, pre-period covariates/CUPED, calendar alignment) and propose a minimal test duration rule. Include a plan for segmentation and multiple-comparison control across 5 planned cuts. D) Outline a decision rubric that ties metric movement to ship/hold/iterate decisions, including what to do if the primary metric is flat but a guardrail regresses.