Explain why CTR rises but CVR unchanged
Company: Amazon
Role: Machine Learning Engineer
Category: Analytics & Experimentation
Difficulty: medium
Interview Round: Onsite
## Experiment analysis (CTR up, CVR flat)
You run an online experiment on an e-commerce product detail page that launches a new UI.
- Primary observation: **CTR increases**, but **purchase conversion rate (CVR) does not change**.
### Questions
1. Using statistical/experimental analysis, how would you investigate *why* CTR improved but CVR did not?
2. Follow-up: You want to analyze the effect by **user tier** (stratified analysis). One tier has **very small sample size**. What statistical approach would you use to still make a reasonable inference?
Quick Answer: This question evaluates a candidate's competency in experimental design and statistical analysis, specifically interpreting divergent engagement and outcome metrics (CTR increase vs CVR unchanged) and performing stratified inference with limited subgroup data.
