Lead a zero-to-one initiative effectively

# 1) Problem Statement - Goal: Increase shopper retention in a way that sustainably improves marketplace health and economics without harming customer experience, partner relationships, or compliance. - Assumptions (explicit): - "Shopper" = independent contractor fulfilling orders. - Retention is measured at multiple horizons (D7, D30, D60, D90) with cohorting by shopper start date. - Marketplace health depends on fill rate, on‑time delivery, shopper earnings/hour, and customer NPS. - Proposed scope v1: Focus on early‑lifecycle retention (first 30–60 days), where hazard of churn is typically highest. # 2) Success Metrics and Guardrails - North star - 60‑day retention of new shoppers (S60): proportion of a start‑month cohort who complete at least one batch in days 31–60. - Formal: S(t) = survival probability to day t. Target: ΔS60 ≥ +3 percentage points within two quarters. - Leading indicators - D7 activation rate: share completing ≥1 batch in first 7 days. - Time‑to‑first‑batch (TTFB): median hours from onboarding complete → first batch. - Early hours worked: median online hours in week 1. - Early experience quality: cancellations, support contacts in first 10 batches. - Economic metrics - Incremental LTV per shopper (ILTV) = Δ(hours worked) × margin/hour. - Cost per retained shopper (CPRS) = incentive + ops cost / incremental retained shoppers. - Marketplace health guardrails (no worse than thresholds vs control): - Order fill rate: ≥ −0.5 pp. - On‑time delivery: ≥ −0.5 pp. - Customer NPS/CSAT: ≥ −1.0 pt. - Shopper earnings/hour: ≥ baseline (no reduction in p50 or p25). - Support contacts/order: ≤ +5%. - Compliance: Pay transparency, classification, and disclosure requirements met. - Budget guardrail: CPRS ≤ $150 (example) with ROI ≥ 1.5× at 6 months. Small numeric example - Baseline: 10,000 new shoppers/month; S60 = 35% → 3,500 retained at D60. - Target: 39% → 3,900 retained; +400 incremental retained. - If each retained shopper contributes 40 orders in next 60 days at $1 margin/order → +$16k gross margin over 60 days. - If incentives/ops cost = $60k → short‑term ROI < 1, but over 6–12 months ILTV may justify; set CPRS and ROI thresholds to ensure sustainability. # 3) Discovery Plan - Quantitative (weeks 1–4) - Build cohort retention curves (D7/D30/D60/D90), survival and hazard analysis. - Segment by cohort month, geo, tenure, device, earnings decile, time‑of‑day, batch type, cancellation, support contacts. - Feature correlation/importance (e.g., SHAP over survival model) to identify drivers: TTFB, idle time, pay variability, batch distance, cancellation exposure, support friction. - Funnel: onboarding complete → first login → first batch → 5th batch → 20th batch; find biggest drop‑offs. - Power analysis for experiments; estimate variance and MDEs. - Qualitative (weeks 1–4) - 1:1 interviews with 20–30 shoppers (new, ramped, churned/reactivated); ride‑alongs/shadowing. - Survey to quantify pain points (clarity of earnings, navigation, batching fairness, tip expectations, payments timing). - Code support tickets for early‑lifecycle themes. - Competitor and policy review - Benchmark payouts cadence, sign‑up bonuses, earnings transparency, guarantees, and compliance constraints. - Synthesize hypotheses - H1: Long TTFB and idle time drive early churn. - H2: Earnings variability and lack of transparency reduce perceived fairness. - H3: Early bad experiences (cancellations, complex substitutions, parking issues) create outsized churn risk. - H4: Payment cadence (slow first payout) weakens reinforcement loop. - Prioritize with RICE/impact vs. effort; run a pre‑mortem (how could this fail?). # 4) PRD Outline (for the initial wedge) - Title: Fast Start for New Shoppers (v1) - Background & Problem - Goals and Non‑Goals - Hypotheses - Target users and segments - User stories (e.g., "As a new shopper, I want my first batch quickly with predictable earnings") - Solution overview - Example components: queue prioritization for first 3 batches, earnings guarantee on first day, real‑time guidance, accelerated first payout. - Requirements - Feature flags, geo eligibility, comms, payments config. - Experiment/rollout plan - Geo‑cluster randomization, holdouts, duration, sample size, MDE. - Metrics & telemetry - Primary, leading, guardrails, dashboards. - Risks & mitigations - Demand displacement, fairness perceptions, legal. - Compliance & Legal checklist - Support/Operations plan - Analytics plan (analysis, HTE, CUPED/diff‑in‑diff as needed) - Launch criteria and kill criteria # 5) Stakeholder Map (RACI exemplar) - Responsible: PM (Shopper Experience), Data Scientist, Eng Lead, Designer, UXR. - Accountable: GM/Director for Supply/Marketplace. - Consulted: Shopper Ops, CX/Support, Trust & Safety, Legal/Compliance, Finance, Marketing/CRM, Dispatch/Matching, Payments, Data Engineering, Partner/Sales. - Informed: Exec sponsor, Regional Ops, Partner Success. # 6) Milestones and Kill/Gate Criteria - Milestones - Week 0–2: Metric definitions finalized; discovery complete; proposal stack‑rank; select v1 wedge. - Week 3–4: PRD v1, experiment design, instrumentation plan; resource/budget approvals. - Week 5–8: Build + internal dogfood; geo pilot launch behind flags. - Week 9–14: Pilot run (6 weeks) with weekly guardrail monitoring. - Week 15–16: Readout; go/no‑go; iterate or scale. - Kill/Gate criteria (examples) - Pre‑pilot: No green light without legal signoff and guardrail dashboards. - During pilot: Immediate pause if any guardrail breach persists >48h (e.g., on‑time −1 pp, earnings/hour −$0.25 at p50) or incident rate spikes. - End of pilot: - Proceed if: ΔS60 ≥ +2 pp (lower bound of 95% CI > +0.5 pp), CPRS ≤ $150, fill rate ≥ −0.3 pp, and NPS ≥ −0.5. - Iterate if: effect positive but below ROI threshold; redesign and retest. - Kill if: effect null/negative or cost/guardrail violations; document and sunset. # 7) De‑risking with Prototype/MVP and Experimentation - MVP concept: "Fast Start" - Queue prioritization for first 3 batches to reduce TTFB. - First‑week earnings guarantee (e.g., $X for first Y hours), reconciled at payout. - Early guidance: checklists, in‑app tips, hotline for first 5 batches. - Accelerated payout for first week. - Low‑lift prototypes - Wizard‑of‑Oz ops: manual priority assignment in a few geos for 2 weeks. - Off‑app comms: SMS nudges, help line, and survey prompts to validate messaging. - Earnings transparency card mock (no ML yet): show conservative expected earnings range. - Experiment design - Geo‑cluster A/B with matched markets; or time‑based randomized windows if interference risk. - CUPED or pre‑period covariate adjustment to reduce variance. - Sample‑size sketch: Detect +3 pp on S30 from 50% baseline (two‑sided, α=0.05, power=0.8) → ~3,200–4,000 shoppers/arm; adjust for clustering and seasonality. - Heterogeneity of treatment effects: tenure, geo, time‑of‑day, demand elasticity. - Interference checks: monitor control‑geo displacement in fill rate. # 8) Obtaining Resources and Budget - One‑pager/PRFAQ: problem, hypotheses, expected ROI, risks, plan, asks. - Business case - Forecast ILTV uplift and CPRS under base/best/worst scenarios. - Show sensitivity to demand, seasonality, and incentive size. - Asks - People: 1 PM, 1 DS, 2–3 Eng, 0.5 DE, 0.5 Designer, UXR support; Ops coverage. - Budget: incentive pool for guarantees/payouts; UXR incentives. - Platform: experiment infra, feature flags, dashboards. - Governance: biweekly steering with GM/Legal/Finance; pre‑reads to accelerate approvals. # 9) Change Management (CX/Legal/Sales/Partners) - CX/Support - Update macros/FAQs; train agents on Fast Start rules and edge cases. - Real‑time escalation path during pilot; monitor contact reasons. - Legal/Compliance - Review incentive structures, disclosures, pay transparency, and classification sensitivities. - Ensure clear T&Cs and opt‑in where needed; regional policy checks. - Sales/Partners - Communicate pilot geos and expected impact on fill/on‑time; align on service levels. - Set expectations: no partner pricing changes; share guardrails and monitors. - Shopper comms - In‑app banners, lifecycle emails/SMS; explain how guarantees work; avoid over‑promising. # 10) Post‑Launch Review and Learning Plan - Timing: interim at 2–3 weeks; full read at 6 weeks; final at 12 weeks. - Contents - Primary effects (S30/S60), leading indicators, guardrails, CPRS, ILTV. - Heterogeneity analysis; external validity; seasonality effects. - Operational learnings (support load, fraud/abuse, gaming of guarantees). - Decision: scale, iterate, or sunset; revised PRD for v2 if applicable. - Documentation: write‑up, dashboard links, decisions log; add to playbook. # 11) 30/60/90‑Day Plan - Days 0–30 - Align definitions; build cohort/survival dashboards; run discovery (quant + qual). - Select v1 wedge (Fast Start); write PRD and experiment design; complete legal review. - Instrumentation plan; power analysis; resource/budget approvals. - Days 31–60 - Build MVP; set up feature flags and monitoring; train CX; finalize comms. - Launch geo‑pilot; weekly reviews; enforce guardrails. - Start concurrent low‑lift tests (nudges, transparency card) for learning velocity. - Days 61–90 - Complete pilot; deep‑dive results and HTE; ROI analysis. - Go/no‑go; scale plan or pivot to next hypothesis (e.g., payment cadence or batching fairness). - Publish learnings; backlog v2; align roadmap for Q+1. # 12) Example Tough Trade‑off - Choice: Broad hourly incentive for all shoppers vs. targeted "Fast Start" for new shoppers. - Broad incentive likely boosts short‑term supply but is expensive and may not improve long‑term retention; risks fairness expectations and demand displacement. - Targeted Fast Start focuses on the highest hazard period with lower CPRS and clearer causal link to retention, but may create perceived inequity with veterans. - Decision: Choose targeted Fast Start to maximize ILTV/ROI, with mitigation: - Communicate purpose clearly; add lightweight recognition for veterans (e.g., loyalty badges, occasional targeted boosts) without undermining economics. # Common Pitfalls and Guardrails - Goodhart’s Law: Don’t optimize D7 at the expense of D60 or earnings/hour. - Selection bias: Ensure randomized or strong quasi‑experimental designs; avoid survivorship bias. - Interference: Geo‑level randomization to avoid spillovers in a shared marketplace. - Seasonality and competitor actions: Use matched‑market controls; extend pilots across periods. - Compliance: Incentives, disclosures, and communications must be legally vetted. # Optional Alternatives if RCT Is Hard - Matched‑market diff‑in‑diff with synthetic controls. - Interrupted time series with guardrails and falsification tests. - Instrumental variables (e.g., exogenous weather shocks) for diagnostics only. This plan turns a vague mandate into a measurable, de‑risked, and cross‑functionally executable path from idea to launch, with explicit metrics, safeguards, and decision gates.