Measure Billboard Campaign Effectiveness and Engagement Quantification
A consumer social platform places billboards in train stations to drive traffic and engagement. You are asked to measure whether the offline campaign worked and how much incremental engagement it caused.
Constraints & Assumptions
-
Treat this as an offline marketing measurement problem where direct user-level attribution is incomplete.
-
Assume data is available for station locations, campaign dates, app/web traffic, sign-ups, sessions, engagement events, revenue, marketing spend, and geography.
-
Account for seasonality, local trends, concurrent campaigns, and selection bias in billboard placement.
-
Engagement should mean incremental behavior caused by exposure, not only traffic from QR codes or vanity URLs.
Clarifying Questions to Ask
-
What business objective did the campaign target: awareness, installs, sign-ups, sessions, purchases, or retention?
-
Which train stations and geographies were treated, and how were they selected?
-
Is there a holdout geography or pre-campaign baseline period?
-
Are direct-response mechanisms such as QR codes, promo codes, or unique URLs available?
Part 1 - Measure Overall Effectiveness
How would you measure the overall effectiveness of the billboard campaign?
What This Part Should Cover
-
Primary outcomes such as incremental sign-ups, app installs, sessions, retained users, purchases, revenue, or return on ad spend.
-
Geo-lift, difference-in-differences, matched-market, or synthetic-control designs.
-
Treatment and control definition, pre-period balance checks, seasonality adjustment, and confidence intervals.
-
Separation of direct attribution from causal incrementality.
Part 2 - Define Attributable Engagement
How would you define and quantify engagement attributable to billboard exposure?
What This Part Should Cover
-
Engagement metrics relevant to the product, such as session starts, content views, saves, follows, searches, shares, comments, or retained activity.
-
Incremental engagement relative to a counterfactual baseline.
-
Use of QR codes, short links, promo codes, survey recall, and geo/time proximity as supporting evidence rather than sole proof.
-
Quality checks such as retention and downstream monetization of acquired users.
Part 3 - Estimate Impacted Population
How would you estimate the size and characteristics of the population impacted by the billboards?
What This Part Should Cover
-
Station foot traffic, commuter overlap, geographic catchment, device/location signals if permitted, survey recall, and demographic or behavioral segments.
-
Reach, frequency, eligibility, and overlap with existing users.
-
Biases in observed exposure data and privacy-safe aggregation.
Part 4 - Forecast Incremental Gains
Propose a framework to forecast potential incremental gains in targeted areas.
What This Part Should Cover
-
Baseline demand, treatment effect estimates, reach assumptions, conversion rates, retention, revenue per user, and cost.
-
Scenario analysis for conservative, base, and optimistic cases.
-
Sensitivity to saturation, repeated exposure, local supply, seasonality, and campaign duration.
What a Strong Answer Covers
A strong answer uses causal geo-based measurement, defines engagement as incremental product value, estimates exposed population carefully, and connects measured lift to forecasted business outcomes and campaign ROI.
Follow-up Questions
-
How would you choose control stations or markets?
-
What would you do if direct QR-code scans are low but geo-level traffic rises?
-
How would you separate the billboard effect from a concurrent digital ad campaign?