Measure YouTube Ad Effectiveness
Company: Uber
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Technical Screen
Uber is running marketing ads on YouTube and wants to understand whether the campaign creates incremental business value, not just whether users watched or clicked the ads.
Answer the following:
1. What success metrics would you use to evaluate the campaign? Include both marketing-funnel metrics and business-outcome metrics.
2. How would you design a measurement approach to estimate the causal effect of the YouTube ads?
3. If you compare treatment and control groups using a t-test, what assumptions must hold?
4. How would you conduct a power analysis before launching the test?
5. Suppose average order value has extreme outliers and the distribution is highly right-skewed. Can you still use a t-test? If not always, what alternatives or adjustments would you consider?
Quick Answer: This question evaluates competencies in causal inference, experimental design, selection of marketing-funnel and business-outcome metrics, statistical testing assumptions, power analysis, and approaches for handling skewed distributions and outliers within analytics.