Regression-adjusted estimation of treatment effects for contribution per order

Context

You are analyzing an A/B test at the order level. The outcome is contribution per order (a continuous, possibly skewed, monetary metric). You have the following variables:

• Treatment indicator T (1 = treated, 0 = control).
• Daypart fixed effects (time-of-day categories).
• Market indicator for Miami vs all other markets.
• Sunday indicator (1 = Sunday, 0 = other days).
• Shopper availability index (continuous, typically at market-day granularity).
• Basket size (use a pre-treatment segment or baseline measure; do not use the realized, post-treatment basket size).
• A treatment × Sunday × Miami interaction of interest.

Answer the following:

1. Model and assumptions

• Specify an OLS (or GLM) model for contribution per order Y that includes: treatment, daypart, market (Miami vs others), shopper availability index, basket size, and treatment × Sunday × Miami interactions. State the identification assumptions needed for unbiased ATE and CATE.

2. Regression vs cohort means

• Compare the regression-adjusted estimator to plain cohort differences (e.g., by Sunday, Miami, basket cohorts) in terms of bias, variance, and interpretability when covariates are imbalanced.

3. Confidence intervals and multiple comparisons

• You’re given three 95% confidence intervals (CIs) for cohort-specific treatment effects (new, repeat–low-basket, repeat–high-basket). Explain what CI overlap does and does not imply about pairwise significance and about an overall treatment effect across cohorts when facing multiple comparisons. Propose a proper multiple-testing correction or a hierarchical model and justify it.