How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a hard difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Roblox.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Roblox during technical interviews.

Determine Player Preference for Local Game Creators

Last updated: Mar 29, 2026

Quick Overview

This question evaluates a data scientist's ability to define primary and secondary engagement and retention metrics that capture player preference for locally created games and to design credible observational causal analyses when randomized experiments are infeasible.

Roblox

Jul 12, 2025, 6:59 PM

Data Scientist

Onsite

Analytics & Experimentation

Player Preference for Local Creators — Metrics and Causal Estimation Without A/B Tests

Context

You are evaluating whether players on Roblox prefer games created by local creators (e.g., creator and player share the same country/region/language). You must:

Define clear primary and secondary metrics that capture “preference.”
If an A/B test is infeasible, propose a credible observational analysis plan that estimates causal effects while mitigating selection bias.

Assume you can label each game session with whether its creator is “local” to the player, and you have standard engagement and retention telemetry at the session and player level.

Tasks

Define primary metric(s) and secondary metric(s) to measure player preference for local creators.
When randomized testing is not possible, outline a causal analysis plan that addresses selection bias (e.g., matching, instrumental variables, difference-in-differences), including assumptions and validation.

Solution

Show

Submit Your Answer

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Roblox•More Data Scientist•Roblox Data Scientist•Roblox Analytics & Experimentation•Data Scientist Analytics & Experimentation