Diagnose conversion-rate time series and CTA swap

Part A — Diagnose a conversion/approval-rate time series

You are given a monthly time series of credit card approval rate from Jan 2013 to Jan 2015. You observe:

• Feb 2013: a noticeable dip
• Nov 2013: a small surge
• From Jan 2014 onward: sustained growth

Assume approval rate is a conversion-like metric:

$\text{Approval Rate} = \frac{\#\text{approved applications}}{\#\text{applications submitted}}$

Questions

1. What are plausible hypotheses for the Feb dip, Nov surge, and the growth starting Jan 2014?
2. What assumptions would you want to validate before drawing conclusions?
3. How would you “go deeper” by analyzing the numerator vs. denominator and explaining why the rate moved?
4. What slices/segments and additional data would you request to validate causality vs. correlation?

Part B — Evaluate swapping two CTAs on a landing page

On an Airwallex webpage there are two primary CTAs (buttons):

• “See a demo”
• “Get started”

A proposal is to swap their positions (e.g., left/right or top/bottom, whichever is more prominent).

Questions

1. What user/business problem could this change be trying to solve, and why might swapping help?
2. How would you design an experiment (or evaluation plan) to test it?
3. Define:
   • a primary success metric
   • diagnostic metrics to explain movement
   • guardrail metrics to prevent harm
4. What confounders/pitfalls (traffic mix, selection bias, novelty effects, downstream lead quality, etc.) would you watch for, and how would you mitigate them?