Define Ultra success metrics and detect suspicious transactions

Quick Overview

This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

Define Ultra success metrics and detect suspicious transactions

Company: TikTok

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: easy

Interview Round: Technical Screen

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data. ## Tables ### `users` - `user_id` BIGINT PRIMARY KEY - `create_date` TIMESTAMP - *(optional if present)* `country` VARCHAR ### `transactions` - `transaction_id` BIGINT PRIMARY KEY - `user_id` BIGINT REFERENCES `users(user_id)` - `transaction_time` TIMESTAMP - `product` VARCHAR -- includes values like `'crypto'`, and may include `'ultra'` if the plan is represented as a product - `amount_gbp` NUMERIC(18,2) - `status` VARCHAR -- `'completed'` / `'declined'` - *(optional if present)* `ip_country` VARCHAR ### `activity` - `user_id` BIGINT REFERENCES `users(user_id)` - `event_time` TIMESTAMP - `product` VARCHAR -- may include `'ultra'` - `event_type` VARCHAR -- `'view'` / `'click'` ## Scenario A: Ultra subscription launch (open-ended) A new **Ultra subscription plan** launches. In a 1-month evaluation window after launch: 1) Define what “success” means with a **primary metric**, **diagnostic metrics**, and **guardrail metrics**. 2) Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them. 3) Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above. ## Scenario B: FinCrime—identify suspicious transactions (open-ended) You partner with the Financial Crime team to flag suspicious behavior. 1) Propose a **rule-based** detection approach that turns “suspicious” into actionable, testable rules using available data. 2) Write example SQL (or pseudo-SQL) that outputs suspicious entities. **Required output for Scenario B** (choose one and state which): - A list of suspicious `user_id`s with a `risk_score` and the reasons/rules triggered, **or** - A list of suspicious `transaction_id`s with flags/reasons. Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.

Quick Answer: This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

|Home/Analytics & Experimentation/TikTok
TikTok logo
TikTok
Feb 4, 2026, 10:59 AM
easyData ScientistTechnical ScreenAnalytics & Experimentation
22
0

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data.

Tables

users

  • user_id BIGINT PRIMARY KEY
  • create_date TIMESTAMP
  • (optional if present) country VARCHAR

transactions

  • transaction_id BIGINT PRIMARY KEY
  • user_id BIGINT REFERENCES users(user_id)
  • transaction_time TIMESTAMP
  • product VARCHAR -- includes values like 'crypto' , and may include 'ultra' if the plan is represented as a product
  • amount_gbp NUMERIC(18,2)
  • status VARCHAR -- 'completed' / 'declined'
  • (optional if present) ip_country VARCHAR

activity

  • user_id BIGINT REFERENCES users(user_id)
  • event_time TIMESTAMP
  • product VARCHAR -- may include 'ultra'
  • event_type VARCHAR -- 'view' / 'click'

Scenario A: Ultra subscription launch (open-ended)

A new Ultra subscription plan launches. In a 1-month evaluation window after launch:

  1. Define what “success” means with a primary metric , diagnostic metrics , and guardrail metrics .
  2. Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them.
  3. Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above.

Scenario B: FinCrime—identify suspicious transactions (open-ended)

You partner with the Financial Crime team to flag suspicious behavior.

  1. Propose a rule-based detection approach that turns “suspicious” into actionable, testable rules using available data.
  2. Write example SQL (or pseudo-SQL) that outputs suspicious entities.

Required output for Scenario B (choose one and state which):

  • A list of suspicious user_id s with a risk_score and the reasons/rules triggered, or
  • A list of suspicious transaction_id s with flags/reasons.

Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.

Loading comments...