Design and justify unread-accounts pinning experiment

You propose a feature that, for users who own multiple accounts (same person_id), pins any accounts with UNREAD notifications to the top of the account switcher. Your goal is to decide whether to launch. Design a concrete experiment and decision framework, then answer the scenario below. Be specific and justify trade-offs: 1) State the primary hypothesis and the target population. Define the unit of randomization and why (e.g., person_id-level cluster vs. account-level), and how you will prevent cross-account contamination when a person has accounts in both variants. 2) Select one Overall Evaluation Criterion (OEC) and 3–5 secondary/diagnostic metrics. For each metric, define the exact numerator/denominator and attribution window (e.g., 'unread resolution rate within 24h', 'cross-account message send rate per person-day', 'account-switch CTR', 'median dwell time on switched-to account', 'notification overload complaints per 1k users', etc.). Explain why the OEC is preferable to simpler clicks/CTR metrics. 3) List 3+ guardrails (e.g., retention, session length, error rate, notification latency, sample ratio mismatch) and the thresholds that would trigger a halt. 4) Specify the exposure and analysis windows (e.g., 28-day run with a 7-day cooldown), novelty/learning effects handling, and how you will handle users who gain/lose accounts mid-experiment. 5) Outline your power analysis inputs and outputs: baseline(s), minimal detectable effect (MDE), variance reduction methods (e.g., CUPED or covariate adjustment), and expected sample size/time to reach 80% power at alpha=0.05. You may assume realistic baselines but must show the formulas or the exact inputs you would plug into a calculator. 6) Data quality and attribution: define 'visit', 'dwell time', 'unread resolved', and how you will attribute outcomes when users switch across web/mobile (cross-device identity). Describe logging you need to add. 7) Decision scenario (interpretation test): Suppose the A/B test shows +40% lift in account-switch CTR to pinned unread accounts, but (a) median dwell time on the switched-to account is <5 seconds, (b) no statistically significant lift in reply/send-message rate or 'unread resolved within 24h', and (c) a +3% increase in bounce-back to the previous account within 10 seconds. Is this a successful outcome under your OEC? Give a yes/no decision and defend it. Propose at least two next steps (e.g., change ranking logic, add richer previews, segment by notification type or user intent) and how you would test them.