Control confounding in observational ad lift
Overview
This question evaluates mastery of observational causal inference for ATE estimation, covering causal DAGs and identification, pre-treatment adjustment, propensity-score and inverse-probability/doubly-robust estimation, diagnostic checks, sensitivity analysis for unobserved confounding, and variance/uncertainty quantification in the context of ad exposure data. It is commonly asked because real-world advertising analyses cannot rely on randomization and interviewers need assurance of both conceptual understanding of identification assumptions and practical application of estimation and diagnostic techniques; the category is Statistics & Math and the level of abstraction spans conceptual reasoning and applied implementation.
Estimating the ATE of Ad Exposure on Conversions (Observational Setup)
You cannot randomize ad exposure. Users differ in age, education, income, and other characteristics. Propose a causal inference plan to estimate the Average Treatment Effect (ATE) of ad exposure on conversions.
Assume we observe users i = 1, …, n over a fixed window. Let A_i ∈ {0,1} indicate whether user i saw the focal ad (at least one impression), and Y_i ∈ {0,1} indicate whether the user converted in the window. Let X_i be pre-exposure covariates.
Include the following:
- A DAG that justifies a valid pre-treatment adjustment set and a brief identification argument.
-
A propensity score model using pre-exposure covariates, and either:
- Matching on the propensity score, or
- Inverse-probability weighting with stabilized weights.
- Formulas for ATE via IPW and doubly robust (AIPW) estimators.
- Diagnostics: overlap checks, standardized mean differences before/after (unweighted and weighted), effective sample size, and weight trimming/clipping.
- Sensitivity analysis for unobserved confounding (e.g., Rosenbaum bounds). Optionally, alternatives like E-values or Oster’s δ.
- How to avoid post-treatment bias (exclude engagement mediators such as clicks/dwell time that occur after exposure).
- Variance estimation and uncertainty reporting (SEs, CIs), including recommendations for sandwich/robust SEs, cluster-robust options, and bootstrap.
Also discuss when you would prefer difference-in-differences (DiD) or CUPED, and the assumptions required.
Solution
Login required