Design and evaluate an uplift model
Overview
This question evaluates competency in causal inference and uplift modeling, experimental design for randomized treatment allocation, offline evaluation of uplift models, operational decision rules under budget constraints, fairness guardrails, and handling interference and deployment of sequential tests in a data-science context.
Targeting a 20% Subset With a Free-Delivery Promotion to Maximize Incremental Orders per Dollar
Context
You work on a two-sided delivery marketplace and want to target at most 20% of customers with a free-delivery promotion. Your objective is to maximize incremental orders per dollar of promotional spend.
Define the causal estimand (individual treatment effect/uplift):
-
uplift(x) = P(Y = 1 | T = 1, X = x) − P(Y = 1 | T = 0, X = x), where:
- Y = 1 if the customer places at least one order in the evaluation window
- T ∈ {0,1} is assignment/exposure to the free-delivery promotion
- X is the customer feature vector
Tasks
(a) Propose an experiment to collect training data, covering randomized holdout, treatment density choices, and compliance tracking.
(b) Propose a modeling approach (e.g., T-/S-/X-/DR-learner or direct uplift models). Describe key features and how you will avoid leakage.
(c) Recommend offline evaluation metrics (Qini, AUUC, uplift-AUC) and a cross-validation strategy.
(d) Specify an on-policy decision rule to choose the top-K customers given a fixed 20% targeting budget and a per-user cost curve for the promotion.
(e) Define guardrails for fairness across geographic zones and user tenure cohorts.
(f) Describe how you will handle interference/spillovers (e.g., delivery capacity constraints, surge) and positivity violations.
Finally, explain how you would deploy and run a sequential test to validate lift without bias from response saturation.
Solution
Login required