Launch Decision Under Asymmetric Costs (Experiment p=0.10)

Context

You ran an A/B test on a churn‑reduction feature. The test's p‑value was 0.10 (suggesting weak evidence of benefit, but not statistically significant at α=0.05). You must decide whether and how to launch, acknowledging asymmetric business costs.

Assume:

• Success threshold: if the true effect is at least −0.15 percentage points monthly churn (i.e., a churn reduction ≥0.15 pp), the annual incremental gross profit is $900k; otherwise $0.
• Ongoing maintenance cost once launched: $240k/year.
• Rollback cost if you reverse a launched feature: $120k (one‑time).
• Prior probability that the true effect meets the threshold (≥ −0.15 pp): 30%.
• Time horizon for NPV: 12 months. State any discounting you assume.

Options

A) Full launch now.

B) 10% phased ramp with sequential monitoring for 6 weeks, then a decision.

C) Stop and redesign for a new test in 8 weeks, then a decision.

Task

For each option:

1. Compute the 12‑month expected NPV, explicitly stating any discounting assumption and any additional testing/decision timing you assume (e.g., when a confirmatory readout happens, and whether rollback occurs upon a negative readout).
2. Include Type I/II error implications at α = 0.05 with reasonable power assumptions you choose and justify.
3. Recommend one option and explain your reasoning.
4. Briefly explain how limited engineering capacity over the next quarter would change your choice or thresholds.