Diagnose conversion-rate time series and CTA swap
Company: Airwallex
Role: Data Scientist
Category: Analytics & Experimentation
Difficulty: easy
Interview Round: Onsite
## Part A — Diagnose a conversion/approval-rate time series
You are given a monthly time series of **credit card approval rate** from **Jan 2013 to Jan 2015**. You observe:
- **Feb 2013:** a noticeable dip
- **Nov 2013:** a small surge
- **From Jan 2014 onward:** sustained growth
Assume approval rate is a conversion-like metric:
\[
\text{Approval Rate} = \frac{\#\text{approved applications}}{\#\text{applications submitted}}
\]
**Questions**
1. What are plausible hypotheses for the Feb dip, Nov surge, and the growth starting Jan 2014?
2. What **assumptions** would you want to validate before drawing conclusions?
3. How would you “go deeper” by analyzing the **numerator vs. denominator** and explaining why the rate moved?
4. What slices/segments and additional data would you request to validate causality vs. correlation?
---
## Part B — Evaluate swapping two CTAs on a landing page
On an Airwallex webpage there are two primary CTAs (buttons):
- **“See a demo”**
- **“Get started”**
A proposal is to **swap their positions** (e.g., left/right or top/bottom, whichever is more prominent).
**Questions**
1. What user/business problem could this change be trying to solve, and why might swapping help?
2. How would you design an experiment (or evaluation plan) to test it?
3. Define:
- a **primary success metric**
- **diagnostic metrics** to explain movement
- **guardrail metrics** to prevent harm
4. What confounders/pitfalls (traffic mix, selection bias, novelty effects, downstream lead quality, etc.) would you watch for, and how would you mitigate them?
Quick Answer: This question evaluates a data scientist's competency in time-series diagnosis, conversion-rate decomposition, causal inference, and A/B experiment design within the Analytics & Experimentation category of Data Science.