Loan Comparison Page: Instrumentation, Metrics, Insights, Experiment, and Cannibalization

Context

You own a loan comparison page (similar to NerdWallet) that lists multiple providers (e.g., SoFi, Upstart). Users can:

• Flow A: Complete an on-site prequalification/application module.
• Flow B: Click out to a partner and complete the flow on the partner site.

You need to measure user behavior and partner performance across both flows, attribute downstream outcomes despite ITP/cookie loss, and design experiments and reporting.

Tasks

1. Instrument end-to-end

• Identify all events and IDs for the page and both flows.
• Specify cross-domain tracking and session stitching to partner outcomes under ITP/cookie loss.

2. Define success metrics and guardrails

• For the page and for each partner, provide precise numerator and denominator definitions (e.g., CTR-to-partner, on-site completion rate, qualified rate, funded-loan rate, revenue per session).

3. Generate three non-obvious insights

• Based on visible differences in partner info (APR ranges, fees, eligibility messaging), hypothesize how user mix and downstream funding could shift.

4. Propose an experiment

• Compare on-site vs click-out entry points for a given partner. Include experimental unit, randomization, stratification, sample-size and power assumptions, a pre-registered decision rule, and approaches to mitigate selection bias and attribution mismatches.

5. Measure cannibalization and contamination

• Outline how to measure partner cannibalization and mixed-traffic contamination across listings (user revisits, tab hoarding, last-click bias). Provide specific event names, join keys, and a minimal metric table schema to drive weekly reviews.