Design Identity-Trust A/B Test

Q: Design Identity-Trust A/B Test

This question evaluates a candidate's competency in experimental design, causal inference, metric selection, sample sizing, bias and confounding identification, interference/network effects, and measurement of low-base-rate adverse events within A/B testing for identity and trust features for a Data Scientist role.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

You are interviewing for a Data Scientist role on an Identity & Trust team at a consumer product company. The team wants to launch a feature that strengthens identity verification and adds trust signals such as a verified badge, additional account checks, or warnings on suspicious profiles.

How would you design an A/B test to evaluate this launch? Discuss:

the unit of randomization and whether user-level randomization is valid,
primary success metrics and guardrail metrics,
how to measure trust when adverse events are rare,
interference or network effects if treated and control users interact,
likely sources of bias or confounding,
how you would size the test and interpret results if fraud decreases but engagement or conversion also declines.

Design Identity-Trust A/B Test

Quick Overview

Solution

Comments (0)