Analyze Comment Counts per Article
You are analyzing the distribution of the number of comments each article receives on a content website. You have comment counts for all articles in a defined period.
Constraints & Assumptions
-
Comment counts are non-negative integers and may be zero-inflated and heavy-tailed.
-
Articles differ in exposure, age, topic, author, placement, and traffic source.
-
Product stakeholders may care about typical article engagement, total comment volume, and whether a UI change increased engagement.
-
Separate descriptive analysis from causal impact analysis.
Clarifying Questions to Ask
-
What time window and article cohort are included?
-
Are counts raw comments, approved comments, unique commenters, or comments per article view?
-
Do we have pageviews, article age, topic, author, and traffic source?
-
Was the UI change randomized, rolled out gradually, or launched to everyone at once?
What a Strong Answer Covers
-
Summary statistics: count, zero rate, mean, median, percentiles, max, variance, standard deviation, IQR, trimmed or winsorized mean, and concentration among top articles.
-
Explanation of why the mean can be misleading when a few viral articles create a long right tail.
-
Exposure-adjusted metrics such as comments per 1,000 views or per active reader.
-
Outlier detection using percentiles, robust z-scores on
log1p(count)
, IQR rules, topic/traffic review, and data-quality checks.
-
Treatment of outliers: investigate first, correct data errors, analyze with and without outliers, cap/winsorize for robustness, or use models suited to heavy-tailed count data.
-
Statistical testing for a UI change: randomized A/B test if available; otherwise difference-in-differences, regression adjustment, permutation/bootstrap tests, or count models such as negative binomial.
-
Guardrails around comment quality, moderation load, spam, and user retention.
Follow-up Questions
-
Would you report mean or median to product managers, and why?
-
How would you compare articles with very different pageviews?
-
What if the UI change increases comments but also increases spam?
-
How would you choose between a t-test, Mann-Whitney test, and negative-binomial regression?