Increase posts receiving comments via experimentation

Quick Overview

This question evaluates a data scientist's skills in experiment and metric design, power/MDE estimation, causal inference with network interference, product analytics, and ideation across supply, demand, matching, and notification levers.

Increase posts receiving comments via experimentation

Company: Meta

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: hard

Interview Round: Technical Screen

Goal: increase the number of posts that receive at least one meaningful comment. a) Define success precisely as the proportion of new posts that receive ≥1 non‑deleted comment within 24 hours of post creation; list guardrails (e.g., DAU, creator session length, abuse reports, comment quality/profanity). b) Establish the current baseline and MDE using the last 28 days; state any seasonality considerations. c) Enumerate ≥10 ideas across supply (make it easier to comment), demand (increase comment intent), matching (surface posts to likely commenters), and notifications/feeds; include risks and expected effect sizes. d) Specify data to collect (post/comment creation times, creator/commenter networks, content type, notifications sent/opened, dwell time). e) Choose your top two ideas and design experiments: unit of randomization, network interference handling (e.g., cluster randomization by poster or community), ramp plan, power assumptions, guardrails, and spillover diagnostics. f) Outline how you’ll attribute lift (incremental commenters vs shifted comments), monitor abuse/quality, and decide ship/no‑ship.

Quick Answer: This question evaluates a data scientist's skills in experiment and metric design, power/MDE estimation, causal inference with network interference, product analytics, and ideation across supply, demand, matching, and notification levers.

|Home/Analytics & Experimentation/Meta
Meta logo
Meta
Oct 13, 2025, 9:49 PM
hardData ScientistTechnical ScreenAnalytics & Experimentation
4
0

Increase the Share of Posts That Receive a Meaningful Comment

You are a data scientist for a consumer social app with posts and comments. Your goal is to increase the fraction of new posts that receive at least one meaningful comment within 24 hours of post creation.

Tasks

(a) Define success precisely as the proportion of new posts that receive ≥1 non-deleted comment within 24 hours of post creation. List appropriate guardrails (e.g., DAU, creator session length, abuse reports, comment quality/profanity).

(b) Establish the current baseline and the minimum detectable effect (MDE) using data from the last 28 days. State any seasonality considerations and how you will account for them.

(c) Enumerate ≥10 product ideas across four levers:

  • Supply (reduce friction to comment)
  • Demand (increase intent to comment)
  • Matching (surface posts to likely commenters)
  • Notifications/feeds (drive timely exposure) For each idea, include risks and expected effect sizes.

(d) Specify the data you will collect to evaluate and iterate (e.g., post/comment creation times, creator/commenter network features, content type, notifications sent/opened, dwell time).

(e) Choose your top two ideas and design experiments for each: unit of randomization, how you will handle network interference (e.g., cluster randomization by poster or community), ramp plan, power assumptions, guardrails, and spillover diagnostics.

(f) Outline how you will attribute lift (incremental commenters vs. shifted comments), monitor abuse/quality, and decide ship/no‑ship.

Loading comments...