Diagnose conversion-rate time series and CTA swap

Quick Overview

This question evaluates a data scientist's competency in time-series diagnosis, conversion-rate decomposition, causal inference, and A/B experiment design within the Analytics & Experimentation category of Data Science.

Diagnose conversion-rate time series and CTA swap

Company: Airwallex

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: easy

Interview Round: Onsite

## Part A — Diagnose a conversion/approval-rate time series You are given a monthly time series of **credit card approval rate** from **Jan 2013 to Jan 2015**. You observe: - **Feb 2013:** a noticeable dip - **Nov 2013:** a small surge - **From Jan 2014 onward:** sustained growth Assume approval rate is a conversion-like metric: \[ \text{Approval Rate} = \frac{\#\text{approved applications}}{\#\text{applications submitted}} \] **Questions** 1. What are plausible hypotheses for the Feb dip, Nov surge, and the growth starting Jan 2014? 2. What **assumptions** would you want to validate before drawing conclusions? 3. How would you “go deeper” by analyzing the **numerator vs. denominator** and explaining why the rate moved? 4. What slices/segments and additional data would you request to validate causality vs. correlation? --- ## Part B — Evaluate swapping two CTAs on a landing page On an Airwallex webpage there are two primary CTAs (buttons): - **“See a demo”** - **“Get started”** A proposal is to **swap their positions** (e.g., left/right or top/bottom, whichever is more prominent). **Questions** 1. What user/business problem could this change be trying to solve, and why might swapping help? 2. How would you design an experiment (or evaluation plan) to test it? 3. Define: - a **primary success metric** - **diagnostic metrics** to explain movement - **guardrail metrics** to prevent harm 4. What confounders/pitfalls (traffic mix, selection bias, novelty effects, downstream lead quality, etc.) would you watch for, and how would you mitigate them?

Quick Answer: This question evaluates a data scientist's competency in time-series diagnosis, conversion-rate decomposition, causal inference, and A/B experiment design within the Analytics & Experimentation category of Data Science.

|Home/Analytics & Experimentation/Airwallex
Airwallex logo
Airwallex
Oct 1, 2025, 12:00 AM
easyData ScientistOnsiteAnalytics & Experimentation
9
0

Part A — Diagnose a conversion/approval-rate time series

You are given a monthly time series of credit card approval rate from Jan 2013 to Jan 2015. You observe:

  • Feb 2013: a noticeable dip
  • Nov 2013: a small surge
  • From Jan 2014 onward: sustained growth

Assume approval rate is a conversion-like metric:

Approval Rate=#approved applications#applications submitted\text{Approval Rate} = \frac{\#\text{approved applications}}{\#\text{applications submitted}}

Questions

  1. What are plausible hypotheses for the Feb dip, Nov surge, and the growth starting Jan 2014?
  2. What assumptions would you want to validate before drawing conclusions?
  3. How would you “go deeper” by analyzing the numerator vs. denominator and explaining why the rate moved?
  4. What slices/segments and additional data would you request to validate causality vs. correlation?

Part B — Evaluate swapping two CTAs on a landing page

On an Airwallex webpage there are two primary CTAs (buttons):

  • “See a demo”
  • “Get started”

A proposal is to swap their positions (e.g., left/right or top/bottom, whichever is more prominent).

Questions

  1. What user/business problem could this change be trying to solve, and why might swapping help?
  2. How would you design an experiment (or evaluation plan) to test it?
  3. Define:
    • a primary success metric
    • diagnostic metrics to explain movement
    • guardrail metrics to prevent harm
  4. What confounders/pitfalls (traffic mix, selection bias, novelty effects, downstream lead quality, etc.) would you watch for, and how would you mitigate them?
Loading comments...