Goal
You need to separate market-driven fluctuations from a product-caused decline in executed_trades per active user around a known release on 2025-07-10.
Setup and Assumptions
-
Metric: executed_trades per active user (daily).
-
Pre window: 2025-06-01–2025-07-09.
-
Post window: 2025-07-10–2025-08-21.
-
You may have staggered exposure (e.g., client versions, geographies) and/or subsets of assets affected.
-
Market factors (e.g., VIX, SPX returns) can strongly influence trading; control for them.
Tasks
-
Design a difference-in-differences (DiD) around the 2025-07-10 release. Define treatment and control groups (e.g., cohorts exposed vs not yet exposed; assets affected vs unaffected; geographies rolling out later). Specify the model equation, fixed effects, and clustered standard errors. State identifying assumptions (parallel trends, no spillovers/no anticipation), and how you will test them. If pre-trends fail, describe a remedy (e.g., synthetic control, matching + staggered DiD, or event study with leads/lags) and why it would be valid.
-
Detect structural breaks and quantify effect size with at least two approaches: one of CUSUM or Bai–Perron multiple change points, and Bayesian Structural Time Series (BSTS). Explain how you will reconcile effect sizes and uncertainty when they disagree.
-
Compute the minimum detectable effect for a 10% decrease in executed_trades per active user with daily aggregation, α = 0.05, power = 0.80, mean active users/day = 200,000, baseline mean = 1.0 trades, SD = 1.5 trades. Use the pooled-variance two-sample t-test formula and state the resulting sample size or window length needed.
-
Propose robustness checks: placebo dates, symbol-level randomization inference, wild bootstrap standard errors, and sensitivity of results to volatility controls (e.g., VIX, SPX return) and holiday dummies. Define pass/fail criteria that would change your decision to ship a fix.