##### Question You are interviewing for a Data Scientist role at a consumer subscription platform (a two-sided social/dating app) that also runs an advertiser business. Work through the following two analytics & experimentation cases. **1. Advertiser-level causal inference with a limited sample** A new advertiser-facing optimization product called **Performance Plus** is being evaluated. The treatment is assigned at the advertiser level, but the pool of eligible advertisers is small. Leadership still wants a credible causal estimate of the product's impact. In your answer, address: 1. The causal estimand and the **primary hypothesis** you would test, plus the counter-hypotheses / failure modes you would consider. 2. The **null and alternative hypotheses**, stated explicitly (and when, if ever, a one-sided test is justified). 3. The **unit of randomization** and why (advertiser vs. campaign vs. geo), given the risk of interference through shared auctions and budgets. 4. Whether you would run a **randomized experiment or a quasi-experimental design** (staggered rollout / difference-in-differences, synthetic control, matched controls, regression discontinuity), and the assumptions each requires. 5. The **primary success metric and guardrail metrics** — e.g. incremental conversions or spend per advertiser, ROAS, CPA, delivery rate, budget exhaustion, advertiser retention — and the trade-offs between noisy ratio metrics and more stable count metrics. 6. How you would **compute power or the minimum detectable effect (MDE)**. 7. How you would **improve power when the advertiser population is limited**: pre-period adjustment / CUPED / ANCOVA, blocking and stratification, matched-pair randomization, panel-data / difference-in-differences with fixed effects, repeated measures, longer duration, sequential testing, hierarchical (partial-pooling) or Bayesian modeling, narrowing the eligible cohort, and choosing a more sensitive metric. 8. The **surprising or counterintuitive findings** you would check for — heterogeneous treatment effects, auction interference, Simpson's paradox / mix shift, survivorship and selection bias, mechanism surprises (e.g. spend up but efficiency down) — and how you would validate they are real rather than noise. **2. Lowering the free-to-paid paywall threshold from 100 to 80 profiles** Free users today hit a paywall after viewing **100 profiles** within the existing quota period. The CEO proposes lowering that threshold to **80 profiles**. How would you decide whether to ship it? Address: 1. The **decision objective** — short-term subscription conversion vs. long-term lifetime value / contribution margin — and why optimizing conversion alone is a trap. 2. The **primary metric, secondary metrics, and guardrails**, balancing monetization against retention, engagement, ad revenue, and user sentiment. 3. The **experiment design**: unit of randomization, target population, segmentation strategy, rollout/ramp, and measurement window. 4. How you would handle **one-sided triggering and post-treatment selection bias** (only users who would browse past 80 are actually exposed), and why a naive triggered-only analysis is invalid. 5. The **risks**: early churn, reduced matching/messaging success, marketplace/network effects, cannibalization of ad revenue, novelty effects, and different impact on new vs. power users. 6. How you would **interpret results and make a ship / no-ship recommendation**, including what you would do if paid conversion rises but retention or engagement declines.