Design Identity & Trust Experiment
Company: Coinbase
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
You are a data scientist at a financial or crypto platform. The product team wants to launch an **Identity & Trust** feature that adds stronger identity verification and a visible trust badge in the transaction flow. The hypothesis is that the feature will reduce fraud, scams, and disputes, while increasing the rate of successful transactions. However, it may also add onboarding friction and hurt conversion.
Design an A/B test to evaluate this feature. In your answer, address:
- the experimental unit and randomization strategy,
- the primary success metric and why it should be primary,
- secondary metrics and guardrails,
- how you would handle interference or spillover if users interact with both treated and untreated counterparties,
- power / minimum detectable effect considerations,
- likely sources of bias or Simpson's paradox across user segments,
- what recommendation you would make if trust and safety improve but conversion declines.
Quick Answer: This question evaluates a data scientist's experimental design and causal inference skills, specifically A/B testing methodology, metric selection, power analysis, interference handling, bias detection, and product analytics for trust-and-safety features.