Implement streaming SRM detector with late events

Quick Overview

This question evaluates proficiency in stateful stream processing, statistical anomaly detection for sample ratio mismatch, and scalable system design concerns such as deduplication, late/out-of-order event handling, real-time aggregation, chi-square testing with Yates correction, bot-rate mitigation, sharding, and replay validation.

Implement streaming SRM detector with late events

Company: TikTok

Role: Data Scientist

Category: Coding & Algorithms

Difficulty: medium

Interview Round: HR Screen

Implement a streaming detector for sample ratio mismatch (SRM) across many concurrent experiments. Input is two topic-partitioned streams: assignments (experiment_id, user_id, variant, ts) and pageviews (user_id, ts). Requirements: 1) deduplicate per (experiment_id, user_id) using idempotent state; 2) maintain rolling counts per variant in O(1) memory per experiment (no raw buffering), supporting late/out-of-order events up to 24 hours; 3) every minute, compute a chi-square goodness-of-fit with Yates correction versus the target split, raise an alert if p < 1e-4 and absolute diff ≥ 0.3 percentage points; 4) guard against bot bursts by excluding users with >N assignments/min; 5) complexity and pseudocode for a single-threaded worker and how you’d shard it; 6) explain how you would validate the detector in replay without leaking ground truth.

Quick Answer: This question evaluates proficiency in stateful stream processing, statistical anomaly detection for sample ratio mismatch, and scalable system design concerns such as deduplication, late/out-of-order event handling, real-time aggregation, chi-square testing with Yates correction, bot-rate mitigation, sharding, and replay validation.

|Home/Coding & Algorithms/TikTok
TikTok logo
TikTok
Oct 13, 2025, 9:49 PM
mediumData ScientistHR ScreenCoding & Algorithms
3
0

Implement a streaming detector for sample ratio mismatch (SRM) across many concurrent experiments. Input is two topic-partitioned streams: assignments (experiment_id, user_id, variant, ts) and pageviews (user_id, ts). Requirements: 1) deduplicate per (experiment_id, user_id) using idempotent state; 2) maintain rolling counts per variant in O(1) memory per experiment (no raw buffering), supporting late/out-of-order events up to 24 hours; 3) every minute, compute a chi-square goodness-of-fit with Yates correction versus the target split, raise an alert if p < 1e-4 and absolute diff ≥ 0.3 percentage points; 4) guard against bot bursts by excluding users with >N assignments/min; 5) complexity and pseudocode for a single-threaded worker and how you’d shard it; 6) explain how you would validate the detector in replay without leaking ground truth.

Submit Your Answer to Earn 20XP

Sign in to leave a comment

Loading comments...