How would you evaluate pixel-issue notifications?

## Context An ads platform supports an **Ads Pixel** (a tracking script) that advertisers install on their websites/apps to send back **conversion events** (e.g., purchases, sign-ups). Pixel issues (broken installation, blocked requests, wrong event schema, delayed/missing events) can degrade measurement and optimization. The product team proposes a new feature in Ads Manager: - The system detects potential **pixel problems** and proactively **notifies advertisers** (in-product and/or email) with suggested fixes. ## Task You are asked to evaluate whether this feature is **good or bad** and whether to **launch** it. ### What you should cover 1. **Define success**: propose a metric framework with - Primary metric(s) - Diagnostic metrics - Guardrail metrics 2. **Design an experiment** (or alternative evaluation strategy if RCT isn’t feasible): - Unit of randomization and eligibility - Treatment/control definition - Duration and sample size / power considerations (high level) - Key segmentation to check heterogeneous effects 3. **Identify risks and confounders**: - Data quality / measurement concerns (pixel data is itself unreliable) - Selection bias (who has a pixel, who sees notifications) - Interference / spillovers (agency-managed accounts, shared pixels) 4. **Explain how you would interpret outcomes** and make a launch decision, including what you’d do if metrics move in opposite directions (e.g., better data quality but worse short-term revenue).