This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Diagnose Sudden Drop in Credit-Card Approval Rate states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
##### Scenario
Fraud Risk – Sudden drop in credit-card approval rate
##### Question
Our approval rate has been stable for months and suddenly dropped in a single day. How would you diagnose the root cause? Outline your investigative steps, data needed, and potential hypotheses.
##### Hints
Consider traffic mix, rule or model changes, data pipeline issues, seasonality, and experiment effects.
Overview: This interview question evaluates metric design, causal reasoning, experiment setup, diagnostics, SQL/statistical checks, and recommendations in a realistic interview setting. A strong answer for Diagnose Sudden Drop in Credit-Card Approval Rate states assumptions, handles edge cases, explains trade-offs, and shows how to validate the result clearly.
You are the on-call Data Scientist supporting the risk/underwriting system. Historically, the daily approval rate has been stable. Yesterday, it dropped sharply in a single day.
Assume you have access to decision logs, rule engine configs, model registry, feature store, experiment platform, observability/monitoring dashboards, and vendor status pages.
Approval rate = approvals / submitted applications, measured on decision-event timestamps. (If your org uses a different definition, state and use it.)
Task
How would you diagnose the root cause? Provide:
A step-by-step investigative plan to localize and explain the drop.
The specific data you would pull.
A set of concrete hypotheses to test (and how to test them).
Hints
Consider:
Traffic mix shifts (channel, merchant/partner, geo, device, new vs. returning).
Rule or model changes (new version, thresholds, reason-code distribution).
Data/feature pipeline issues (null spikes, late features, schema changes).