How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at TikTok.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at TikTok during technical interviews.

Investigate Declining ROI and Propose Effective Solutions

Quick Overview

This question evaluates a data scientist's skills in advertising effectiveness measurement, causal inference, experiment design, and diagnostic analytics for two-sided marketplaces, covering north-star and guardrail metrics, attribution, and root-cause analysis for declining ROI.

E-commerce Ads Effectiveness and Diagnostics (Analytics & Experimentation)

Context

You are a data scientist on an e-commerce platform responsible for measuring advertising effectiveness and diagnosing performance issues across a two-sided marketplace (users and merchants). You work with paid placements/ads that affect merchant GMV and ad ROI.

Tasks

Resume Project Walkthrough

Describe one relevant project end-to-end:
- Objectives and success metrics (north-star and supporting/guardrail metrics).
- How you evaluated success (incrementality, attribution, and statistical evaluation).
- Key details of your A/B experiment design (unit of randomization, power/MDE, duration, interference handling, guardrails).

Case 1 — Grow Merchant Ad Revenue or Merchant GMV

The business wants to increase either merchant advertising revenue (ad spend on the platform) or merchant GMV. How would you:
- Frame objectives and select metrics.
- Analyze current performance (funnel, segments, cohorts).
- Propose data-driven actions and experiments.

Case 2 — ROI on Ads Has Declined

ROI/ROAS has recently fallen. Walk through how you would:
- Diagnose root causes with a structured metrics tree and analyses (segmentation, cohorts, time-series, experiment logs).
- Recommend short-term mitigations and longer-term fixes.

Consider Including

Clear north-star metric and supporting metrics.
Experiment setup for marketplaces with potential auction/interference effects.
Funnel, cohort, and segmentation analyses.
A hypothesis-driven root-cause framework and validation plan.

Quick Overview

Tasks

Resume Project Walkthrough

Describe one relevant project end-to-end:

Objectives and success metrics (north-star and supporting/guardrail metrics).
How you evaluated success (incrementality, attribution, and statistical evaluation).
Key details of your A/B experiment design (unit of randomization, power/MDE, duration, interference handling, guardrails).

Case 1 — Grow Merchant Ad Revenue or Merchant GMV

The business wants to increase either merchant advertising revenue (ad spend on the platform) or merchant GMV. How would you:

Frame objectives and select metrics.
Analyze current performance (funnel, segments, cohorts).
Propose data-driven actions and experiments.

Case 2 — ROI on Ads Has Declined

ROI/ROAS has recently fallen. Walk through how you would:

Diagnose root causes with a structured metrics tree and analyses (segmentation, cohorts, time-series, experiment logs).
Recommend short-term mitigations and longer-term fixes.

Investigate Declining ROI and Propose Effective Solutions

Quick Overview

E-commerce Ads Effectiveness and Diagnostics (Analytics & Experimentation)

Context

Tasks

Consider Including

Solution

Submit Your Answer to Earn 20XP

Investigate Declining ROI and Propose Effective Solutions

Quick Overview

E-commerce Ads Effectiveness and Diagnostics (Analytics & Experimentation)

Context

Tasks

Consider Including

Solution

Submit Your Answer to Earn 20XP