How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Pinterest.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Pinterest during technical interviews.

Investigate Homepage Experiment Without Control Group: Methods and Metrics

Q: Investigate Homepage Experiment Without Control Group: Methods and Metrics

This question evaluates proficiency in causal inference for non-randomized treatments, experiment design hygiene, product-metric definition for a new module, and diagnostic segmentation for root-cause analysis.

Scenario

A social-media homepage team is running experimentation and product-metric analyses on a personalized feed. An intern accidentally launched a treatment to a user cohort without a randomized control group. You have standard event logs (user_id, timestamps), impression/click events, pre-period behavior history, device/geo/app-version, and eligibility flags.

Tasks

No-control experiment: How can you still estimate treatment impact? Compare matching versus propensity-score weighting and discuss trade-offs.
A/B test hygiene: Review an existing randomized A/B test and list common pitfalls that could bias the results.
New module metrics: For a new horizontal home-feed module, what primary metrics would you track to judge success?
Diagnostic scenario: Post-launch, you observe homepage click-through rate (CTR) dropping in treatment while DAU and time spent stay flat. How would you investigate root causes, and which user segments would you analyze first?

Hints

Causal inference when randomization is absent (matching, propensity scores, DiD/ITS/synthetic control).
Experiment design pitfalls and guardrails.
Metrics hierarchy: module-level, session-level, ecosystem, and health.
Segmentation: device, app version, geography, tenure, power vs casual users, exposure/reach to the module.

Scenario

Tasks

No-control experiment: How can you still estimate treatment impact? Compare matching versus propensity-score weighting and discuss trade-offs.

A/B test hygiene: Review an existing randomized A/B test and list common pitfalls that could bias the results.

New module metrics: For a new horizontal home-feed module, what primary metrics would you track to judge success?

Diagnostic scenario: Post-launch, you observe homepage click-through rate (CTR) dropping in treatment while DAU and time spent stay flat. How would you investigate root causes, and which user segments would you analyze first?

Hints

Causal inference when randomization is absent (matching, propensity scores, DiD/ITS/synthetic control).

Experiment design pitfalls and guardrails.

Metrics hierarchy: module-level, session-level, ecosystem, and health.

Segmentation: device, app version, geography, tenure, power vs casual users, exposure/reach to the module.

Investigate Homepage Experiment Without Control Group: Methods and Metrics

Quick Overview

Scenario

Tasks

Hints

Solution

Comments (0)

Investigate Homepage Experiment Without Control Group: Methods and Metrics

Quick Overview

Scenario

Tasks

Hints

Solution

Comments (0)