Evaluates causal inference and time-series experimentation skills—specifically handling confounding, staggered rollouts, seasonality, and credible treatment-effect identification—in the Analytics & Experimentation domain at a mid-to-senior data scientist abstraction level.
You work on a Roblox-like game platform. A new product change ("feature") is rolled out and you want to estimate its causal impact on user engagement, measured as daily time spent (minutes per user-day).
However, the rollout is not fully randomized:
You have event-level logs aggregated to a user-day table:
user_id
date
(in UTC)
minutes_spent
feature_enabled
(1 if the feature is enabled for that user on that date)
country
,
platform
,
account_age_days
)
Task:
minutes_spent
, including how you would handle
confounding
.
State any additional assumptions you need and what outputs (tables/plots) you would show to stakeholders.