Determine if players prefer local creators without experiments

Q: Determine if players prefer local creators without experiments

This is a Analytics & Experimentation interview question from Roblox for Data Scientist roles. View the full question and solution on PracHub.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

You cannot run an A/B test. Define “local creators” as creators whose games use a player’s native language. Design an observational study to estimate whether players prefer local creators. Be specific: 1) State the primary metric and justify it versus at least one plausible alternative (e.g., time-per-session vs total time vs 7-day retention vs conversion to spend). 2) Specify the unit of analysis, sessionization rules, and how you’ll handle multi-language or multi-region games and players who play both local and non-local titles. 3) Propose a causal identification strategy (e.g., propensity score matching, difference-in-differences, or IV). Write down the estimand you would identify (e.g., ATT) and the identification assumptions. 4) List concrete covariates for matching/balancing, propose a caliper/overlap check, and the exact balance diagnostics you’ll report. 5) Suggest at least one credible instrument or quasi-experiment (e.g., staggered language-localization releases, regional outages), with validity threats and falsification/placebo tests. 6) Describe robustness checks (e.g., alternative metrics, session caps, winsorization), heterogeneity cuts (market, device, player tenure), and how you would interpret a null result. 7) Provide a minimal schema you’d need and 2–3 SQL snippets or pseudocode to compute the primary metric and treatment indicator accurately.

Determine if players prefer local creators without experiments

Comments (0)