Design an experiment assignment service

Q: Design an experiment assignment service

This question evaluates system-design and analytics competencies for online experimentation platforms in the Analytics & Experimentation domain, focusing on deterministic sticky assignment, configuration storage, eligibility targeting, traffic allocation, mutual exclusion, overrides, exposure logging, metric tagging, privacy/compliance, latency and availability targets, experiment lifecycle, and high-level architecture. It is commonly asked because it assesses both conceptual understanding and practical system-design application of distributed-system trade-offs, scalability, data integrity, operational reliability, and privacy constraints, and gauges the ability to define high-level APIs, architecture, and justify key trade-offs.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Q: What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Visa.

Q: What role is this question designed for?

This question is commonly asked for Software Engineer candidates at Visa during technical interviews.

Question

Design an Online Experimentation (A/B Testing) Assignment Service

You are asked to design a highly available online experiment assignment service. The service exposes an API that takes (userId, featureName) and returns a variant identifier (e.g., control or treatment). Assume the service will be used by web and mobile applications globally and must support both logged-in and logged-out users.

Clarify, specify, and propose designs for the following:

Deterministic sticky assignment and bucketing strategy
Configuration storage for experiments and features
Eligibility targeting and audience filters
Traffic allocation and ramps
Mutual exclusion and experiment layering
Overrides (holdouts, QA, killswitch)
Exposure logging and idempotency
Metric tagging and data pipeline
Privacy and compliance (e.g., GDPR/CCPA)
Latency/availability targets and fallbacks
Experiment lifecycle (create, start, pause, stop, analyze)
High-level architecture and key components

Provide a high-level architecture, key APIs, and justify key trade-offs. Where relevant, include small examples and precise definitions (e.g., hashing and bucket math, namespace/layering).

Design an experiment assignment service

Quick Overview

Design an experiment assignment service

Design an Online Experimentation (A/B Testing) Assignment Service

Write your answer