How would you evaluate causal lift and paywall?

You are interviewing for a Data Scientist role at a consumer subscription platform with an advertiser business. Answer the following two analytics cases. 1. Advertiser-level causal inference study A new advertiser optimization product called Performance Plus is being evaluated. The treatment is assigned at the advertiser level, but the number of eligible advertisers is limited. You need to estimate the causal impact of the product on advertiser outcomes. Please explain: - What primary hypothesis you would test, and what counter-hypotheses or failure modes you would consider. - Whether you would run a randomized experiment or use a quasi-experimental design, and why. - How you would ensure enough statistical power when the advertiser population is small. Discuss minimum detectable effect, variance reduction, stratification, matched-pair randomization, CUPED or pre-period adjustment, panel-data or difference-in-differences approaches, sequential testing, and when Bayesian or hierarchical modeling may help. - What primary metric and guardrail metrics you would use. Consider metrics such as advertiser spend, conversions, cost per acquisition, return on ad spend, delivery rate, budget exhaustion, and advertiser retention. - What surprising findings or heterogeneous treatment effects you would look for after the analysis, and how you would validate that they are real rather than noise. 2. Consumer monetization case Today, free users can browse up to 100 profiles before they are pushed toward a paid experience. The CEO wants to reduce that threshold from 100 profiles to 80 profiles. Please explain: - How you would evaluate whether this change should launch. - What success metric, guardrail metrics, and decision criteria you would use, balancing short-term monetization against long-term retention and user experience. - What experiment design, target population, segmentation strategy, and measurement window you would choose. - What risks you would watch for, including novelty effects, one-sided triggering, selection bias, cannibalization of ad revenue, churn, and negative user sentiment. - What recommendation you would make if paid conversion increases but retention or engagement declines.