Design Identity & Trust Experiment

You are a data scientist at a financial or crypto platform. The product team wants to launch an Identity & Trust feature that adds stronger identity verification and a visible trust badge in the transaction flow. The hypothesis is that the feature will reduce fraud, scams, and disputes, while increasing the rate of successful transactions. However, it may also add onboarding friction and hurt conversion.

Design an A/B test to evaluate this feature. In your answer, address:

• the experimental unit and randomization strategy,
• the primary success metric and why it should be primary,
• secondary metrics and guardrails,
• how you would handle interference or spillover if users interact with both treated and untreated counterparties,
• power / minimum detectable effect considerations,
• likely sources of bias or Simpson's paradox across user segments,
• what recommendation you would make if trust and safety improve but conversion declines.