Estimate sales impact from reviews causally
Overview
This question evaluates a data scientist's competency in causal inference, experimental design, observational modeling, and statistical reporting for quantifying the impact of product reviews on sales, including defining treatments and outcomes and addressing key confounders.
Your PM asks: Do better product reviews cause higher sales, or do higher sales lead to more reviews? Design an analysis to estimate the causal effect of reviews on sales. Deliverables: 1) define outcome(s) and treatment(s) (e.g., average rating, review count, recent reviews, helpful votes) and the time windows; 2) enumerate key confounders (seasonality, promotions, price changes, stockouts, product life cycle) and how you’ll control for them; 3) propose an experiment (e.g., vary review visibility/order or badging) with primary metrics, guardrails, and sample size calculations; 4) if experimentation is infeasible, propose a quasi-experimental design (difference-in-differences using staggered review-widget rollout, or regression discontinuity at visibility thresholds like the first visible star) and a complementary observational model (panel regression with product and time fixed effects, lagged ratings, and instrument ideas); 5) specify checks for reverse causality, selection bias, and survivorship; 6) show how you would report the effect size (elasticity: % sales change per +0.1 rating) with confidence intervals and how to validate robustness.