How would you test billboard effectiveness?
Company: Airwallex
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
Pinterest is considering an offline billboard advertising campaign in selected cities to increase brand awareness and downstream user growth. As the data scientist, design the measurement plan.
Address the following:
- How would you design the billboard campaign and define success metrics? Consider awareness, traffic, conversion, and guardrail metrics.
- How would you test effectiveness? Discuss an ideal randomized geo experiment and what you would do if randomization is not feasible.
- Explain the difference between Propensity Score Matching (PSM) and Propensity Score Weighting (PSW or IPW). What estimand does each target, and what does each method effectively optimize for?
- How would you calculate sample size or minimum detectable effect for this study? State the experimental unit, the main formula, and any adjustments needed for clustered or geo-based experiments.
- What tradeoffs and failure modes would you watch for, such as spillovers, seasonality, selection bias, delayed effects, and budget constraints?
Quick Answer: This question evaluates experimental design, causal inference, statistical power/sample-size calculation, and marketing measurement competencies relevant to offline advertising effectiveness within the Analytics & Experimentation domain for data scientists.