How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Technical Screen rounds at Amazon.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Amazon during technical interviews.

Diagnose Causes and Test Hypotheses for Metric Drop

Last updated: Mar 29, 2026

Quick Overview

This question evaluates diagnostic analytics, causal reasoning, instrumentation and data-quality troubleshooting skills along with practical experiment design and statistical power considerations for a Data Scientist role in the Analytics & Experimentation domain.

Amazon

Jul 12, 2025, 6:59 PM

Data Scientist

Technical Screen

Analytics & Experimentation

Scenario

A large consumer web/mobile product sees its key business metric drop materially and suddenly.

Assume: The sitewide purchase conversion rate fell by 12% relative (e.g., 10.0% → 8.8%) starting Tuesday 10:00 AM and persisting for several days.

Task

List plausible causes for the performance drop.
For each cause, describe specific analyses you would run to validate or falsify it (be concrete about cuts, comparisons, and supporting logs/metadata).
Choose your main hypothesis and design an A/B test to confirm it, including:
- Hypothesis and treatment/control design
- Randomization unit and exposure
- Sample size, duration, and analysis plan
Specify primary and guardrail metrics you would track and why.

Hints

Use segmentation (e.g., device, geo, channel, app version), funnel breakouts, external factors, and a metric hierarchy (north-star vs. input vs. health).
Consider data quality and release/flag timelines alongside user behavior.

Solution

Show

Comments (0)

Loading comments...

Browse More Questions

More Analytics & Experimentation•More Amazon•More Data Scientist•Amazon Data Scientist•Amazon Analytics & Experimentation•Data Scientist Analytics & Experimentation