Define Ultra success metrics and detect suspicious transactions

Q: Define Ultra success metrics and detect suspicious transactions

This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data.

Tables

`users`

user_id BIGINT PRIMARY KEY
create_date TIMESTAMP
(optional if present) country VARCHAR

`transactions`

transaction_id BIGINT PRIMARY KEY
user_id BIGINT REFERENCES users(user_id)
transaction_time TIMESTAMP
product VARCHAR -- includes values like 'crypto' , and may include 'ultra' if the plan is represented as a product
amount_gbp NUMERIC(18,2)
status VARCHAR -- 'completed' / 'declined'
(optional if present) ip_country VARCHAR

`activity`

user_id BIGINT REFERENCES users(user_id)
event_time TIMESTAMP
product VARCHAR -- may include 'ultra'
event_type VARCHAR -- 'view' / 'click'

Scenario A: Ultra subscription launch (open-ended)

A new Ultra subscription plan launches. In a 1-month evaluation window after launch:

Define what “success” means with a primary metric , diagnostic metrics , and guardrail metrics .
Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them.
Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above.

Scenario B: FinCrime—identify suspicious transactions (open-ended)

You partner with the Financial Crime team to flag suspicious behavior.

Propose a rule-based detection approach that turns “suspicious” into actionable, testable rules using available data.
Write example SQL (or pseudo-SQL) that outputs suspicious entities.

Required output for Scenario B (choose one and state which):

A list of suspicious user_id s with a risk_score and the reasons/rules triggered, or
A list of suspicious transaction_id s with flags/reasons.

Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.