Decide goal under ambiguity

# Decision Framework and Solution ## 0) Core principle: optimize the largest LTV lever - Subscription LTV can be approximated as LTV ≈ (ARPU × gross margin) / monthly churn. - Sensitivity: a relative increase in churn reduces LTV roughly one-for-one; a relative increase in ARPU increases LTV one-for-one (holding other terms constant). Choose the goal where small, feasible improvements yield the largest LTV lift within constraints and time. ## 1) Leading indicators and falsifiable hypotheses Leading indicators (pre-build or rapid prototyping signals that map to each goal): - LI-1 (Retention): Return-next-week rate among targeted subscribers after exposure to the benefit (adopters vs. matched non-adopters). Why: early habit formation predicts lower churn. - LI-2 (Acquisition): Paywall CTR and trial-start uplift when the benefit is messaged on paywalls/landing pages (A/B vs. control). - LI-3 (ARPU): Upsell attach rate to the benefit (opt-in rate at a proposed price point) and incremental ARPU among exposed vs. control, with cancellation-due-to-price as a guardrail. Falsifiable hypotheses (clear thresholds and time windows): - H1 (Retention): Among high-risk subscribers (bottom 3 deciles of engagement), exposing the benefit increases 28-day return rate by ≥8% relative (e.g., from 50% to 54%) and reduces 60-day churn by ≥10% relative vs. control. If not observed, we reject retention as the primary goal. - H2 (Acquisition): Showing the benefit on the paywall increases trial-start rate by ≥10% relative (e.g., 1.2% to 1.32%) with no ≥3% relative drop in trial→paid conversion. If either fails, we reject acquisition as the primary goal. - H3 (ARPU): Pricing the benefit at $X yields ≥8% attach rate among eligible subscribers and ≥3% net ARPU lift for the exposed cohort, with cancellations due to price not exceeding +0.2 percentage points vs. control. If not met, we reject ARPU as the primary goal. ## 2) Minimum data required and plan if data are missing Minimum metrics and definitions: - Retention/churn: - Monthly gross churn (% of paying subs who cancel in month). Also 30/60/90-day cohort survival. - Early retention proxies: 7- and 28-day return rates, weekly active days per subscriber, success actions per week tied to the benefit. - Acquisition funnel: - Visitor→paywall view, paywall CTR, trial-start rate, trial→paid conversion, paid→month-2 retention. - Revenue/ARPU: - ARPU by cohort and segment, attach rate to add-ons, discount/share of revenue on promo, cancellation reason codes (esp. price-related), gross margin if available. - Exposure/adoption: - Benefit exposure flag, adoption flag, time-to-first-use, frequency of benefit use. Time windows: - Baseline trend: last 8–12 weeks to understand seasonality. - Experiment readouts: early indicators at 7 and 28 days; primary retention at 60 days; ARPU at 30/60 days; acquisition immediately (daily) and stable at 2 weeks. Key segments: - Tenure: new (0–3 months), maturing (3–12), established (>12). - Plan type/tier and price cohort (full price vs. discounted). - Geography and platform (iOS/Android/Web). - Acquisition channel (paid, organic, referral). - Engagement risk deciles (from a churn model or heuristic usage buckets). If crucial data are missing: - Instrument fast: add exposure/adoption events, paywall experiment IDs, cancellation reason capture within 1 week. - Use proxies: if 60-day churn is unavailable, use 28-day return and DAU/MAU as leading indicators; if ARPU granularity is missing, track add-on attach and average order value. - Run a 1–2 week pre-flight A/A to validate event quality; delay high-impact decisions until basic telemetry is reliable. - Choose a reversible path: gate the benefit behind feature flags, start with small, high-signal cohorts (e.g., high-risk churners) to learn cheaply. ## 3) Choice given the provided signals Signals: churn rose from 4.8% to 5.4% (two months), conversion is flat at 1.2%, ARPU up 3% from price mix. LTV sensitivity: - Churn increase: 4.8% → 5.4% is a +12.5% relative increase in churn, implying roughly −12.5% LTV if ARPU unchanged. - ARPU increase: +3% relative, implying +3% LTV if churn unchanged. - Net effect: approximately −9.5% LTV. The churn deterioration dominates. Crisp choice: prioritize Retention as the primary goal. - Rationale: The relative churn increase materially outweighs the ARPU lift; acquisition is flat. Protecting the existing base has the largest, fastest LTV impact and reduces future acquisition pressure. The benefit should be designed and targeted to drive repeat use and habit formation among at-risk subscribers first. Secondary goals/guardrails: - Guard ARPU: do not erode price integrity; avoid discounts that mask retention gains. - Watch acquisition: ensure paywall messaging of the benefit does not depress conversion. ## 4) Risks, mitigation, and rollback plan Key risks of picking the wrong primary goal: - If retention is wrong: we underinvest in paywall positioning that could have accelerated growth; opportunity cost on top-of-funnel. - If acquisition is wrong: benefit and messaging might attract low-LTV users, raising churn and support costs. - If ARPU is wrong: paywalls/upsells could increase churn via price sensitivity, netting negative LTV. Mitigation and rollout plan (dates and kill criteria): - Week 0–1 (Instrumentation & Baseline): finalize events, set up feature flag, define cohorts; confirm 8–12 weeks of baseline metrics are stable. - Week 2–3 (Alpha, internal and 1% high-risk cohort): validate UX, event fidelity, and LI-1 return-next-week uplift; halt if data quality <95% completeness. - Week 4–7 (A/B Beta, 10–20% of target segments): - Primary success: ≥8% relative lift in 28-day return; projected ≥10% relative reduction in 60-day churn for high-risk cohort (H1). - Guardrails: - Churn guardrail: no increase ≥0.3 percentage points in 30-day churn vs. control at 95% confidence. - ARPU guardrail: no net ARPU decline ≥1% vs. control; cancellations due to price +0.2pp or less. - Acquisition guardrail (if messaging paywall): trial-start rate not down ≥5% and trial→paid not down ≥3% relative. - Support guardrail: tickets per 1k subs not up ≥30%. - Week 8 (Decision checkpoint): - Roll forward to 50–100% of high-risk segments if primary and guardrails met; expand to broader segments only after confirming 60-day churn deltas. - If H1 not met or any guardrail breached, rollback via feature flag within 24 hours, revert paywall copy, and publish a post-mortem. Pivot goal: if LI-2 showed strong uplift with stable retention, consider switching primary to Acquisition with a new test plan; if LI-3 strong and churn neutral, consider ARPU. Kill criteria (terminate/pivot): - Failure to meet ≥2 of 3 hypothesis thresholds by Week 7, or any guardrail breach as specified above at 95% confidence, or negative net LTV impact estimate (ARPU × margin vs. churn) > −2%. Contingencies: - If seasonality suspected (e.g., holidays), extend beta by 2 weeks but keep cumulative exposure <25% of base. - Maintain communication and reversibility: all changes behind flags; pre-approved rollback copy; clear owner and 24/7 on-call during experiments. ## Summary - Use LTV sensitivity to decide: current signals point clearly to Retention as the primary goal. - Validate with leading indicators tied to falsifiable hypotheses before scaling. - Require minimal but sufficient telemetry; if missing, instrument fast and use proxies. - Stage-gate with explicit guardrails, kill criteria, and rapid rollback to limit downside if the chosen goal is wrong.