Evaluate causal power and paywall change

Answer the following two analytics interview cases for a Data Scientist role. 1. **Advertiser causal inference with a limited sample.** You are evaluating an advertiser-facing optimization product called **Performance Plus**. The eligible advertiser population is small, but leadership wants a credible causal estimate of the product's impact. How would you ensure the study has enough statistical power? In your answer, specify: - the unit of randomization and why; - the primary success metric(s) and guardrail metric(s), for example ROAS, CPA, spend, conversions, and advertiser retention; - how you would compute power or minimum detectable effect (MDE); - how you would improve power when the advertiser population is limited, for example pre-period adjustment, blocking/stratification, longer duration, repeated-measures designs, or Bayesian/sequential approaches; - the null and alternative hypotheses; - what surprising or counterintuitive findings you would check for after the experiment, such as heterogeneous treatment effects, interference, Simpson's paradox, or survivorship bias. 2. **Lowering the free-to-paid threshold.** On a subscription app, free users currently hit a paywall after viewing **100 profiles** within the existing product-defined quota period. The CEO proposes lowering the threshold to **80 profiles**. How would you evaluate whether this change is good for the business? Discuss: - the decision objective, such as short-term subscription conversion versus long-term user lifetime value; - primary metrics, guardrails, and possible trade-offs; - experiment design, segmentation, and rollout strategy; - risks such as churn, reduced matching success, marketplace/network effects, and different impact on new users versus power users; - how you would interpret the results and make a ship / no-ship recommendation.