Evaluate concession gift-card policy with DID
Company: Amazon
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: hard
Interview Round: Onsite
Some regions piloted a concession program: when a shipment is lost/damaged, affected customers receive a gift card instead of just refund/replacement. Evaluate the program’s impact. First, clarify objectives to pick outcomes (e.g., star ratings, complaint rates, repeat purchase, churn, CSAT, NPS, cost per incident). Inventory data availability and unit of analysis (customer vs city) and whether pre‑pilot data exist. Specify a DID with staggered adoption: write the regression (treatment indicator × post, unit and time fixed effects, covariates), clustering choice, and an event‑study for pre‑trend checks. If parallel trends fails, propose remedies and selection‑on‑observables alternatives (PSM‑DID with common support and balance checks), or synthetic control at the city level; justify when each is appropriate. Detail guardrail metrics (refund costs, abuse, customer service load), heterogeneity (severity, item price), and cost‑benefit. If you could run an RCT, design it (unit, randomization, stratification, sample size/power, spillover controls) and define stop/go decision rules.
Quick Answer: This question evaluates a data scientist's competency in causal inference and experimentation design, encompassing difference-in-differences with staggered adoption and event-study diagnostics, data inventory and unit-of-analysis specification, clustering choices, heterogeneity and guardrail analyses, and cost–benefit and optional randomized experiment design. Commonly asked in Analytics & Experimentation interviews because it gauges the ability to justify identification strategies, diagnose violations of parallel trends and propose alternatives, and translate empirical results into business-impact metrics, it sits at the intersection of conceptual understanding and practical application within the Data Science domain.
