Evaluates Facebook Group health metrics and threaded-comments experiment design. Strong answers define normalized health scores, compare small and large groups fairly, and test deep interaction with safety guardrails.
Scenario: Groups range from intimate hobby clubs to massive public communities. Leadership wants a single ‘health’ score, while engineers plan to launch Reddit‑style threaded comments. You need to pick health metrics, compare small vs large Groups, and validate whether the new UI improves deep discussion.
Question 1: Which metrics would you track to measure Group health? (Hint: share of active members, lurker ratio, violation rate)
Question 2: How would you compare performance between large and small Groups? (Hint: scale‑normalised engagement per capita)
Question 3: If Reddit‑style comment threads are introduced, how would you test for interaction lift? (Hint: A/B design, retention, depth of discussion)
Overview: Evaluates Facebook Group health metrics and threaded-comments experiment design. Strong answers define normalized health scores, compare small and large groups fairly, and test deep interaction with safety guardrails.
Group Health Metrics and Threaded Comments Experiment
You are a Data Scientist working on a platform with Groups ranging from small hobby clubs to very large public communities. Leadership wants a comparable "Group health" score across Groups. Engineering plans to launch Reddit-style threaded comments to encourage deeper discussions.
Constraints & Assumptions
Group health is multidimensional.
Comparisons between small and large Groups must be normalized.
Threaded comments may change engagement depth, moderation load, and member experience.
Include experiment design and guardrails.
Clarifying Questions to Ask Guidance
What types of Groups are in scope?
Is the goal retention, meaningful discussion, safety, growth, or creator/admin satisfaction?
How are posts, comments, reactions, reports, and moderation actions logged?
Can threaded comments be randomized by group?
Part 1 - Health Metrics
Which metrics would you track to measure Group health?
What This Part Should Cover Guidance
Include active member share, contributor rate, posts, comments, reactions, retention, growth, response rate, discussion depth, safety reports, moderation burden, and member satisfaction.
Include quality and diversity of participation, not only volume.
Define metrics over a consistent time window.
Part 2 - Fair Comparison Across Group Sizes
How would you compare small and large Groups fairly?
What This Part Should Cover Guidance
Use per-member rates, percentiles, empirical Bayes shrinkage, size bands, topic benchmarks, and confidence intervals.
Avoid ranking small Groups solely on noisy raw rates.
Normalize by age, category, public/private status, and activity opportunity.
Part 3 - Threaded Comments Experiment
How would you test whether threaded comments increase deep interaction?
What This Part Should Cover Guidance
Randomize at group or cluster level to avoid within-group contamination.