Non-Randomized Launch Decision via Synthetic Control: Complete Analysis Plan

You need to make a go/no-go decision for a high-impact feature that cannot be A/B tested due to policy/infrastructure constraints. Assume we can gate the feature to one or a small number of units (e.g., a geography × platform cell) and measure time-series KPIs across comparable units.

Propose a complete analysis plan using Synthetic Control (or justify an alternative if SCM is unsuitable). Address the following:

(a) Treatment unit and donor pool construction

• Define the treatment unit precisely.
• Specify how you will build the donor pool.
• Eligibility/exclusion rules to prevent contamination, spillovers, and anticipation effects.

(b) Pre-intervention window and external factors

• Pre-period length and rationale.
• How you will handle seasonality, holidays, macro shocks, and data latency/backfill.

(c) Metrics and decision rubric

• Primary outcome and guardrail metrics.
• Pre-registered success thresholds and a clear go/no-go decision rule.

(d) Predictors, weights, and tuning

• Which predictors to include (outcome lags vs. covariates).
• Weight constraints and how you will tune hyperparameters.

(e) Diagnostics and inference

• Required diagnostics before trusting effects (e.g., pre-period RMSPE targets, placebo tests in-space/in-time, pre/post fit plots).
• Inference approach (MSPE ratio/permutation tests, uncertainty bands for pointwise and cumulative effects).

(f) Sensitivity and spillovers

• Sensitivity analyses (leave-one-out donors, alternative windows, augmented/regularized SCM, donor reweighting).
• How you will bound and assess spillovers/contamination.

(g) Heterogeneity and persistence

• Subgroup and dynamic-effect analyses to assess heterogeneity and persistence/decay.

(h) Fallbacks

• What you will do if pre-period fit is poor or donors are scarce.

(i) Launch mapping

• How results translate into a staged launch plan, rollback criteria, and post-launch monitoring.