How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at TikTok.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at TikTok during technical interviews.

Test 15s to 60s video length change | TikTok Interview Question

Q: Test 15s to 60s video length change

This question evaluates a data scientist's competency in experimental design, causal inference, metric definition and instrumentation, power/MDE calculations, interference and contamination handling on multi‑sided platforms, and rollout planning.

Experiment Design: Change Default Creator Video Length from 15s to 60s

Context

You are designing an A/B test for a short‑form video platform with two sides: creators (supply) and viewers (demand). Product proposes changing the default preselected recording length in the creation UI from 15 seconds to 60 seconds. The hypothesis is that this nudge will increase the supply and consumption of longer videos, affecting viewer engagement, monetization, and system health.

Task

Design the experiment and cover the following:

Unit of randomization and exposure rules to avoid contamination (creators, viewers, or geo; how to handle multi‑sided interactions).
Primary outcomes and guardrails (e.g., completion rate, total watch time per user, creator retention, session length, ad revenue per mille, latency/crash rate), with concrete metric definitions and instrumentation.
Traffic allocation, ramp schedule, and expected MDE with power assumptions.
How to detect novelty and learning effects and set experiment duration.
Checks for SUTVA/interference and cluster variance.
A rollout plan if results pass.

Context

Task

Design the experiment and cover the following:

Unit of randomization and exposure rules to avoid contamination (creators, viewers, or geo; how to handle multi‑sided interactions).

Primary outcomes and guardrails (e.g., completion rate, total watch time per user, creator retention, session length, ad revenue per mille, latency/crash rate), with concrete metric definitions and instrumentation.

Traffic allocation, ramp schedule, and expected MDE with power assumptions.

How to detect novelty and learning effects and set experiment duration.

Checks for SUTVA/interference and cluster variance.

A rollout plan if results pass.

Test 15s to 60s video length change

Quick Overview

Experiment Design: Change Default Creator Video Length from 15s to 60s

Context

Task

Solution

Comments (0)

Test 15s to 60s video length change

Quick Overview

Experiment Design: Change Default Creator Video Length from 15s to 60s

Context

Task

Solution

Comments (0)