Design pre-launch plan and cluster A/B test

A Facebook feature ('More like this' button that surfaces similar products) is being considered for Instagram, but it has not launched on Instagram. You only have access to the interactions and products schema described above (no Instagram button telemetry yet) and cannot use revenue as the primary goal; the goal is user engagement. Devise a concrete plan to convince the PM the feature is necessary before launch and to validate it after launch: 1) Pre-launch evidence using existing data: define one or two leading indicators (e.g., exploration rate = sessions with at least one similar-product interaction / sessions, session depth, or per-user interaction_count with similar items) you can estimate without the button via observational analysis. Describe a causal approach to reduce bias (e.g., propensity score weighting/matching using buyer history, product category, country; difference-in-differences using products that already show similar carousels in other entry points). Specify the exact covariates you would include and how you’d validate overlap and balance. 2) Experiment design: pick the randomization unit and justify it under network effects. For example, choose country- or social-graph clusters to mitigate interference between friends; explain how you’d form clusters, estimate ICC, and handle cross-border users. Define control (no button) vs treatment (button enabled) precisely. 3) Primary/secondary metrics: choose engagement-focused primary metrics (not revenue), guardrails (e.g., feed latency p95, crash rate, seller complaints, non-engagement regressions such as add-to-cart rate), and detailed event logging you’d add at launch to measure exposure and intent (impressions, clicks, dwell, saves). 4) Power and runtime: outline how you’d compute sample size and MDE under cluster randomization, including assumptions you need (baseline, variance, ICC, desired power/alpha). Describe how you’d handle sequential looks (e.g., alpha-spending or group sequential designs) and define a minimum runtime to cover weekly seasonality. 5) Decision and rollout: specify exact launch criteria that combine statistical significance and practical significance, what analyses you would run for heterogeneity (e.g., by country, new vs returning users) with multiple-testing control, and how you’d stage rollout if results are promising but not uniformly positive.