Measure Billboard Campaign Effectiveness and Engagement Quantification
Quick Overview
Evaluates offline marketing measurement for a billboard campaign, including how to define attributable engagement, estimate exposed population, and forecast incremental traffic, sign-ups, retention, revenue, and ROI. Strong answers use geo-lift, matched markets, or synthetic controls rather than direct attribution alone.
Measure Billboard Campaign Effectiveness and Engagement Quantification
Company: TikTok
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
##### Scenario
Offline marketing: Pinterest places billboards in train stations to drive traffic and engagement.
##### Question
How would you measure the overall effectiveness of this billboard campaign? Define and quantify ‘engagement’ attributable to the billboard exposure. How can you estimate the size and characteristics of the population impacted by the billboards? Propose a framework to forecast potential incremental gains (sign-ups, sessions, revenue) in targeted areas.
##### Hints
Consider geo-lift, difference-in-differences, synthetic controls, tracking URLs/QR codes, baseline trends.
Quick Answer: Evaluates offline marketing measurement for a billboard campaign, including how to define attributable engagement, estimate exposed population, and forecast incremental traffic, sign-ups, retention, revenue, and ROI. Strong answers use geo-lift, matched markets, or synthetic controls rather than direct attribution alone.
Measure Billboard Campaign Effectiveness and Engagement Quantification
A consumer social platform places billboards in train stations to drive traffic and engagement. You are asked to measure whether the offline campaign worked and how much incremental engagement it caused.
Constraints & Assumptions
Treat this as an offline marketing measurement problem where direct user-level attribution is incomplete.
Assume data is available for station locations, campaign dates, app/web traffic, sign-ups, sessions, engagement events, revenue, marketing spend, and geography.
Account for seasonality, local trends, concurrent campaigns, and selection bias in billboard placement.
Engagement should mean incremental behavior caused by exposure, not only traffic from QR codes or vanity URLs.
Clarifying Questions to Ask Guidance
What business objective did the campaign target: awareness, installs, sign-ups, sessions, purchases, or retention?
Which train stations and geographies were treated, and how were they selected?
Is there a holdout geography or pre-campaign baseline period?
Are direct-response mechanisms such as QR codes, promo codes, or unique URLs available?
Part 1 - Measure Overall Effectiveness
How would you measure the overall effectiveness of the billboard campaign?
What This Part Should Cover Guidance
Primary outcomes such as incremental sign-ups, app installs, sessions, retained users, purchases, revenue, or return on ad spend.
Geo-lift, difference-in-differences, matched-market, or synthetic-control designs.
Treatment and control definition, pre-period balance checks, seasonality adjustment, and confidence intervals.
Separation of direct attribution from causal incrementality.
Part 2 - Define Attributable Engagement
How would you define and quantify engagement attributable to billboard exposure?
What This Part Should Cover Guidance
Engagement metrics relevant to the product, such as session starts, content views, saves, follows, searches, shares, comments, or retained activity.
Incremental engagement relative to a counterfactual baseline.
Use of QR codes, short links, promo codes, survey recall, and geo/time proximity as supporting evidence rather than sole proof.
Quality checks such as retention and downstream monetization of acquired users.
Part 3 - Estimate Impacted Population
How would you estimate the size and characteristics of the population impacted by the billboards?
What This Part Should Cover Guidance
Station foot traffic, commuter overlap, geographic catchment, device/location signals if permitted, survey recall, and demographic or behavioral segments.
Reach, frequency, eligibility, and overlap with existing users.
Biases in observed exposure data and privacy-safe aggregation.
Part 4 - Forecast Incremental Gains
Propose a framework to forecast potential incremental gains in targeted areas.
What This Part Should Cover Guidance
Baseline demand, treatment effect estimates, reach assumptions, conversion rates, retention, revenue per user, and cost.
Scenario analysis for conservative, base, and optimistic cases.
Sensitivity to saturation, repeated exposure, local supply, seasonality, and campaign duration.
What a Strong Answer Covers Guidance
A strong answer uses causal geo-based measurement, defines engagement as incremental product value, estimates exposed population carefully, and connects measured lift to forecasted business outcomes and campaign ROI.
Follow-up Questions Guidance
How would you choose control stations or markets?
What would you do if direct QR-code scans are low but geo-level traffic rises?
How would you separate the billboard effect from a concurrent digital ad campaign?