This question evaluates skills in causal inference and quasi-experimental design, including specifying estimands, choosing RD and staggered DiD approaches, accounting for treatment intensity, and applying robustness checks and spillover detection when measuring policy impacts on abuse and business KPIs.

A traffic throttling policy was launched for sellers flagged as risky. Because the rollout lacked a clean A/B test, you must estimate the causal impact of throttling on abuse and business KPIs using observational data. Primary outcomes include:
(a) Specify the causal estimands (e.g., average treatment effect on treated sellers) for both outcomes.
(b) Propose two credible quasi-experimental designs and when each is valid:
(c) Define the analysis window, unit of observation, and how to handle treatment intensity (dose = percent rank penalty or impression share reduced).
(d) List at least three robustness/validity checks (e.g., placebo thresholds, negative-control outcomes, falsification on pre-policy periods, sensitivity to trimming top-GMV sellers, bounding bias from misclassification).
(e) Explain how you would detect and adjust for spillovers (e.g., seller migration to new accounts) and survivorship bias.
Login required