Prove source growth is cannibalization, not incremental
Overview
This question evaluates causal inference and experimental-design competencies in analytics, testing skills in identification strategy, treatment and control definition, specification of difference‑in‑differences or randomized geo experiments, metric construction, power calculations, and robustness checks for attribution and cannibalization analysis. It is commonly asked to determine whether observed growth is incremental or substitution-driven, falls under the Analytics & Experimentation domain in Data Science, and emphasizes practical application accompanied by conceptual understanding of identification assumptions, statistical power, and measurement.
Causal Analysis Design: Is Web Growth Incremental or Cannibalization?
Background
You observe that revenue attributed to creation_source = "web" is higher in 2026 compared to 2025. You need to determine whether this increase is primarily incremental or reflects cannibalization/substitution from other acquisition sources (api, mobile).
Assume revenue is tracked weekly by geo and source, and budgets can be adjusted by source at the geo level.
Task
Design a causal analysis that tests whether web growth is primarily cannibalization rather than incremental revenue. Address the following:
- Identification
- Choose and justify an approach (e.g., randomized geo budget shift, difference-in-differences with matched geos, or synthetic control).
- Define treatment (e.g., reducing non-web budgets by X% while holding web constant, or vice versa), control units, and the time windows (pre, treatment, washout).
- Model
- Specify the core difference-in-differences (DID) equation with unit and time fixed effects and an interaction capturing treatment, plus controls for seasonality, macro trends, advertiser mix, and product changes.
- State key assumptions (parallel trends, no spillovers) and how you’ll test pre-trends and interference (e.g., cluster or partial interference models).
- Metrics
- Define incremental revenue and the substitution/cannibalization rate = −ΔRevenue_other_sources / +ΔRevenue_web within randomized units.
- Specify confidence intervals, unit of analysis (geo, advertiser, cohort), and aggregation.
- Power
- Provide a minimal detectable effect (MDE) calculation given historical variance and sample size (number of geos/advertisers and weeks). A worked example is acceptable.
- Robustness
- Plan placebo tests on pre-periods, negative-control outcomes, alternative specifications (event study, synthetic control), and sensitivity to heterogeneous treatment effects.
- Decision Rule
- Pre-specify thresholds on cannibalization rate and incremental ROI that would trigger budget reallocation.
- Provide a mocked analysis-plan table schema you’d need (unit_id, date, source, revenue, spend, cohort, geo, treatment_flag).
Solution
Login required