How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Affirm.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Affirm during technical interviews.

Diagnose Sudden Drop in Credit-Card Approval Rate

Quick Overview

This question evaluates a data scientist's competency in root-cause analysis for production metric anomalies, testing diagnostic reasoning across decision logs, model and rule changes, feature pipelines, traffic-mix shifts, experiment assignments, and external dependencies.

Fraud Risk: Sudden Drop in Credit Approval Rate

Context

You are the on-call Data Scientist supporting the risk/underwriting system. Historically, the daily approval rate has been stable. Yesterday, it dropped sharply in a single day.

Assume you have access to decision logs, rule engine configs, model registry, feature store, experiment platform, observability/monitoring dashboards, and vendor status pages.

Approval rate = approvals / submitted applications, measured on decision-event timestamps. (If your org uses a different definition, state and use it.)

Task

How would you diagnose the root cause? Provide:

A step-by-step investigative plan to localize and explain the drop.
The specific data you would pull.
A set of concrete hypotheses to test (and how to test them).

Hints

Consider:

Traffic mix shifts (channel, merchant/partner, geo, device, new vs. returning).
Rule or model changes (new version, thresholds, reason-code distribution).
Data/feature pipeline issues (null spikes, late features, schema changes).
Seasonality/calendar anomalies and promotions.
Experiment/AB-test effects (assignments, guardrails, SRM).
External dependencies (KYC/identity, bureau pulls, device fingerprinting, payment networks).
Infrastructure/latency/timeouts and fail-closed policies.

Quick Overview

Context

You are the on-call Data Scientist supporting the risk/underwriting system. Historically, the daily approval rate has been stable. Yesterday, it dropped sharply in a single day.

Assume you have access to decision logs, rule engine configs, model registry, feature store, experiment platform, observability/monitoring dashboards, and vendor status pages.

Approval rate = approvals / submitted applications, measured on decision-event timestamps. (If your org uses a different definition, state and use it.)

Hints

Consider:

Traffic mix shifts (channel, merchant/partner, geo, device, new vs. returning).

Rule or model changes (new version, thresholds, reason-code distribution).

Data/feature pipeline issues (null spikes, late features, schema changes).

Seasonality/calendar anomalies and promotions.

Experiment/AB-test effects (assignments, guardrails, SRM).

External dependencies (KYC/identity, bureau pulls, device fingerprinting, payment networks).

Infrastructure/latency/timeouts and fail-closed policies.

Diagnose Sudden Drop in Credit-Card Approval Rate

Quick Overview

Fraud Risk: Sudden Drop in Credit Approval Rate

Context

Task

Hints

Solution

Comments (0)

Diagnose Sudden Drop in Credit-Card Approval Rate

Quick Overview

Fraud Risk: Sudden Drop in Credit Approval Rate

Context

Task

Hints

Solution

Comments (0)