PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/TikTok

Define Ultra success metrics and detect suspicious transactions

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

  • easy
  • TikTok
  • Analytics & Experimentation
  • Data Scientist

Define Ultra success metrics and detect suspicious transactions

Company: TikTok

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: easy

Interview Round: Technical Screen

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data. ## Tables ### `users` - `user_id` BIGINT PRIMARY KEY - `create_date` TIMESTAMP - *(optional if present)* `country` VARCHAR ### `transactions` - `transaction_id` BIGINT PRIMARY KEY - `user_id` BIGINT REFERENCES `users(user_id)` - `transaction_time` TIMESTAMP - `product` VARCHAR -- includes values like `'crypto'`, and may include `'ultra'` if the plan is represented as a product - `amount_gbp` NUMERIC(18,2) - `status` VARCHAR -- `'completed'` / `'declined'` - *(optional if present)* `ip_country` VARCHAR ### `activity` - `user_id` BIGINT REFERENCES `users(user_id)` - `event_time` TIMESTAMP - `product` VARCHAR -- may include `'ultra'` - `event_type` VARCHAR -- `'view'` / `'click'` ## Scenario A: Ultra subscription launch (open-ended) A new **Ultra subscription plan** launches. In a 1-month evaluation window after launch: 1) Define what “success” means with a **primary metric**, **diagnostic metrics**, and **guardrail metrics**. 2) Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them. 3) Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above. ## Scenario B: FinCrime—identify suspicious transactions (open-ended) You partner with the Financial Crime team to flag suspicious behavior. 1) Propose a **rule-based** detection approach that turns “suspicious” into actionable, testable rules using available data. 2) Write example SQL (or pseudo-SQL) that outputs suspicious entities. **Required output for Scenario B** (choose one and state which): - A list of suspicious `user_id`s with a `risk_score` and the reasons/rules triggered, **or** - A list of suspicious `transaction_id`s with flags/reasons. Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.

Quick Answer: This question evaluates a data scientist's ability to define primary, diagnostic, and guardrail metrics for a product launch and to design rule-based suspicious-transaction detection using available schema, emphasizing metric definition, defensible proxy selection, SQL-based data modeling, and financial-crime domain knowledge.

Related Interview Questions

  • Plan DS approach for biker delivery project - TikTok (easy)
  • Define and critique a user activity metric - TikTok (easy)
  • Design and decompose Trust & Safety risk metrics - TikTok (easy)
  • Analyze promo anomaly and design risk guardrails - TikTok (Medium)
  • Design an interference-robust A/B test for monetization - TikTok (hard)
TikTok logo
TikTok
Feb 4, 2026, 10:59 AM
Data Scientist
Technical Screen
Analytics & Experimentation
5
0

You work on a fintech product with these existing tables (UTC timestamps). You may only use these tables/columns; if a metric cannot be measured directly, you must propose a defensible proxy using available data.

Tables

users

  • user_id BIGINT PRIMARY KEY
  • create_date TIMESTAMP
  • (optional if present) country VARCHAR

transactions

  • transaction_id BIGINT PRIMARY KEY
  • user_id BIGINT REFERENCES users(user_id)
  • transaction_time TIMESTAMP
  • product VARCHAR -- includes values like 'crypto' , and may include 'ultra' if the plan is represented as a product
  • amount_gbp NUMERIC(18,2)
  • status VARCHAR -- 'completed' / 'declined'
  • (optional if present) ip_country VARCHAR

activity

  • user_id BIGINT REFERENCES users(user_id)
  • event_time TIMESTAMP
  • product VARCHAR -- may include 'ultra'
  • event_type VARCHAR -- 'view' / 'click'

Scenario A: Ultra subscription launch (open-ended)

A new Ultra subscription plan launches. In a 1-month evaluation window after launch:

  1. Define what “success” means with a primary metric , diagnostic metrics , and guardrail metrics .
  2. Describe key risks to validity (e.g., seasonality, marketing campaigns, selection bias) and how you’d mitigate them.
  3. Write example SQL (or pseudo-SQL) to compute your chosen metrics using only the tables above.

Scenario B: FinCrime—identify suspicious transactions (open-ended)

You partner with the Financial Crime team to flag suspicious behavior.

  1. Propose a rule-based detection approach that turns “suspicious” into actionable, testable rules using available data.
  2. Write example SQL (or pseudo-SQL) that outputs suspicious entities.

Required output for Scenario B (choose one and state which):

  • A list of suspicious user_id s with a risk_score and the reasons/rules triggered, or
  • A list of suspicious transaction_id s with flags/reasons.

Call out false-positive/false-negative tradeoffs and how you would evaluate and iterate.

Solution

Show

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More TikTok•More Data Scientist•TikTok Data Scientist•TikTok Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.