Determine Key Metrics for Spend-Tracker Launch Decision
Overview
This question evaluates the candidate's competency in A/B testing, experiment analysis, segmentation, metric selection, and business-oriented cost–benefit reasoning, and it sits in the Analytics & Experimentation domain of Data Science while testing practical application supported by conceptual understanding.
Chime
Aug 4, 2025, 10:55 AM
Data Scientist
Technical Screen
Analytics & Experimentation
52
0
A/B Decision Framework: Spend-Tracker Feature
Context
A fintech app ran an A/B test (Test vs. Control) on a new Spend-Tracker feature. Results are split by two user segments:
- High-Income
- Non-High-Income
Available metrics include:
- Average revenue (per user)
- Total revenue
- Profit
- Acquisition cost
Task
Decide whether to launch the Spend-Tracker and in which segments. Specifically:
- State the factors, calculations, and decision thresholds you would use.
- Identify the two or three metrics you would prioritize and explain why.
- Explain how you would interpret and act on divergent results between High-Income and Non-High-Income users.
- List additional analyses or data you would request before a final launch decision.
Hints: Compare relative lift vs cost, test for statistical significance, check segment interaction effects, and weigh long-term LTV against acquisition spend.
Solution
Login required
Show