PracHub
QuestionsPremiumLearningGuidesCheatsheetNEWCoaches
|Home/Analytics & Experimentation/Roku

Measure speaker impact without A/B testing

Last updated: Mar 29, 2026

Quick Overview

This question evaluates causal inference, observational study design, metric specification, instrumental variable reasoning, handling of interference and seasonality, and basic power/MDE calculations in a data science analytics context.

  • Medium
  • Roku
  • Analytics & Experimentation
  • Data Scientist

Measure speaker impact without A/B testing

Company: Roku

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: Medium

Interview Round: Technical Screen

Your company launched an optional smart speaker on 2025-01-10. Adoption is voluntary; 6% of existing app users purchase within 60 days, and you cannot force adoption or run an A/B test. Available data: user-level app logs, device graph, CRM attributes, ad exposures, prices, shipments, and regional stockouts. A) Define and justify one primary engagement metric and guardrails (e.g., churn, substitution to other devices). B) Design two complementary identification strategies to estimate the causal effect on engagement at +30 and +180 days: (1) staggered-adoption difference-in-differences/event-study; (2) propensity-score matched IPW/DR or a synthetic control at the user/region level. For each, write the estimand, key assumptions, diagnostics (pre-trends, covariate balance, negative controls), and how you will handle interference (household spillovers) and seasonality. C) Leverage exogenous variation from regional stockouts/shipping delays as an instrument: argue relevance and exclusion, and outline falsification tests. D) Provide a back-of-the-envelope power/MDE given 6% adoption and historical variance: what sample size or horizon is needed to detect a 3% lift?

Quick Answer: This question evaluates causal inference, observational study design, metric specification, instrumental variable reasoning, handling of interference and seasonality, and basic power/MDE calculations in a data science analytics context.

Related Interview Questions

  • Diagnose coffee-shop profit decline - Roku (Medium)
  • Analyze Profit Decline: Data Collection and Hypothesis Testing - Roku (medium)
  • Measure Speaker's Impact Using Propensity Score Matching - Roku (medium)
Roku logo
Roku
Oct 13, 2025, 9:49 PM
Data Scientist
Technical Screen
Analytics & Experimentation
3
0

Your company launched an optional smart speaker on 2025-01-10. Adoption is voluntary; 6% of existing app users purchase within 60 days, and you cannot force adoption or run an A/B test. Available data: user-level app logs, device graph, CRM attributes, ad exposures, prices, shipments, and regional stockouts. A) Define and justify one primary engagement metric and guardrails (e.g., churn, substitution to other devices). B) Design two complementary identification strategies to estimate the causal effect on engagement at +30 and +180 days: (1) staggered-adoption difference-in-differences/event-study; (2) propensity-score matched IPW/DR or a synthetic control at the user/region level. For each, write the estimand, key assumptions, diagnostics (pre-trends, covariate balance, negative controls), and how you will handle interference (household spillovers) and seasonality. C) Leverage exogenous variation from regional stockouts/shipping delays as an instrument: argue relevance and exclusion, and outline falsification tests. D) Provide a back-of-the-envelope power/MDE given 6% adoption and historical variance: what sample size or horizon is needed to detect a 3% lift?

Comments (0)

Sign in to leave a comment

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Roku•More Data Scientist•Roku Data Scientist•Roku Analytics & Experimentation•Data Scientist Analytics & Experimentation
PracHub

Master your tech interviews with 7,500+ real questions from top companies.

Product

  • Questions
  • Learning Tracks
  • Interview Guides
  • Resources
  • Premium
  • For Universities
  • Student Access

Browse

  • By Company
  • By Role
  • By Category
  • Topic Hubs
  • SQL Questions
  • Compare Platforms
  • Discord Community

Support

  • support@prachub.com
  • (916) 541-4762

Legal

  • Privacy Policy
  • Terms of Service
  • About Us

© 2026 PracHub. All rights reserved.