How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Roku.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Roku during technical interviews.

Measure Speaker's Impact Using Propensity Score Matching

Quick Overview

This question evaluates a data scientist's skills in causal inference for observational studies, focusing on identification strategy, confounding control, handling staggered adoption, estimator selection (e.g., propensity-score approaches and event-study difference-in-differences), diagnostics, and sensitivity analysis.

Causal Impact of Optional Device Ownership on Engagement (Observational Design)

Scenario

A new, optional smart speaker is released. Some users purchase and link the speaker to their account; others do not. Users self-select into purchasing, and you cannot run a forced A/B test.

Assume you have: (a) user-level panel data (daily/weekly) on engagement (e.g., streaming hours, sessions), (b) a time-stamped indicator of speaker activation/ownership, (c) rich covariates (prior engagement levels and trends, demographics/geo, devices, marketing exposure), and (d) staggered adoption timing across users.

Task

How would you measure the causal impact of owning the speaker on engagement? Describe your preferred design and why.

In your answer, specify:

Identification strategy and main assumptions.
How you would construct the control group and handle staggered adoption.
Model/estimator choice and diagnostics (e.g., pre-trend checks).
Sensitivity analyses and alternatives if assumptions fail.

You may discuss quasi-experimental approaches such as difference-in-differences (event study), propensity-score matching/weighting, instrumental variables, and pre-post checks.

Quick Overview

Scenario

A new, optional smart speaker is released. Some users purchase and link the speaker to their account; others do not. Users self-select into purchasing, and you cannot run a forced A/B test.

Task

How would you measure the causal impact of owning the speaker on engagement? Describe your preferred design and why.

In your answer, specify:

Identification strategy and main assumptions.

How you would construct the control group and handle staggered adoption.

Model/estimator choice and diagnostics (e.g., pre-trend checks).

Sensitivity analyses and alternatives if assumptions fail.

You may discuss quasi-experimental approaches such as difference-in-differences (event study), propensity-score matching/weighting, instrumental variables, and pre-post checks.

Measure Speaker's Impact Using Propensity Score Matching

Quick Overview

Causal Impact of Optional Device Ownership on Engagement (Observational Design)

Scenario

Task

Solution

Comments (0)

Measure Speaker's Impact Using Propensity Score Matching

Quick Overview

Causal Impact of Optional Device Ownership on Engagement (Observational Design)

Scenario

Task

Solution

Comments (0)