Diagnose a Structural Break in KYC Submission Rate

A weekly KYC submission-rate chart was stable, then showed a sudden step down on a specific date, continued to decline gradually afterward, and displayed a short-lived increase near the end of each year. Develop a structured investigation. Separate the three patterns, propose plausible mechanisms, identify the data needed to distinguish them, and specify the next analyses or experiments. The goal is to determine whether the chart reflects a real product problem, a measurement problem, or changes in the population entering the funnel. ### Clarifying Questions to Ask - What are the exact numerator, denominator, eligibility rules, and time zone? - Is the chart based on signup date, KYC attempt date, or outcome date? - Did the rate change because the numerator fell, the denominator rose, or both? - Were product, policy, vendor, or tracking changes deployed near the break? ### Part 1 — Sudden structural break Investigate the sharp level change at one date. #### What This Part Should Cover - Instrumentation and metric-pipeline validation before causal stories. - Release, UX, compliance-policy, provider, and acquisition changes near the breakpoint. - Segments and unaffected controls that can localize the mechanism. ### Part 2 — Gradual decline Determine whether the continued downward trend is within segments or caused by traffic and merchant mix. #### What This Part Should Cover - Decomposition of overall rate into segment shares and segment-specific rates. - Cohort, country, device, campaign, and merchant-type views with stable definitions. - Tests for denominator quality, maturation lag, and changing intent or risk mix. ### Part 3 — Recurring year-end increase Explain how you would evaluate the repeated seasonal bump without confusing seasonality with an intervention effect. #### What This Part Should Cover - Comparison with prior years and matched calendar periods. - Holiday operations, campaign mix, merchant demand, and processing-lag hypotheses. - A model or design that separates trend, seasonality, and discrete changes. ```hint Decompose before explaining Plot numerator and denominator volumes, then compare weighted segment rates with a counterfactual holding segment shares fixed. ``` ### What a Strong Answer Covers - Distinct hypothesis sets for a step change, slow trend, and recurring seasonality. - Concrete falsification tests and ownership of the required data. - Metric integrity, delayed outcomes, and composition effects. - A prioritized sequence rather than an unbounded list of possible causes. ### Follow-up Questions - How would you estimate the impact if one UX release occurred exactly at the breakpoint? - What decomposition quantifies how much of the decline comes from mix shift? - Which leading indicators would you monitor while KYC outcomes mature?