Instant Book: Metrics, Measurement, Rollout, and Risk Plan
Context
You are evaluating an "Instant Book" feature that allows customers to immediately book a pro for a pre-set price and time, without waiting for quotes. Design a comprehensive plan to measure, roll out, and govern the feature across a two-sided marketplace.
Tasks
-
North-star and objectives
-
Propose one north-star metric for marketplace health and two secondary objectives (one demand-side, one supply-side).
-
Provide exact formulas and units for each metric.
-
Diagnostic metrics
-
List at least six diagnostics covering: liquidity (time to first commit), fulfillment rate, cancellations/reschedules, refund rate, average booking value, pro utilization, and fairness (e.g., Gini of bookings across pros within category–region–week).
-
For each, specify target direction and what short-term regression (if any) is acceptable during ramp.
-
Measurement plan
-
Define instrumentation/events, deduping rules, and attribution windows.
-
Explain how you will separate organic bookings from Instant Book, handle multi-device sessions, and prevent double-counting when a request also receives quotes.
-
Rollout strategy
-
Recommend a staged launch (e.g., supply-gated by category–region with minimum active pros and SLOs), plus a holdback for long-term effects.
-
Describe quota controls to avoid starving non–Instant Book requests.
-
Success criteria and decision tree
-
Specify quantitative launch gates after 2 and 6 weeks, including guardrails that must not worsen by more than X%.
-
Risks and mitigations
-
Identify at least five risks (e.g., adverse selection, schedule conflicts, price anchoring, unfair exposure, fraud) and propose concrete mitigations and monitoring.