Design metrics and experiment for donation feature

Q: Design metrics and experiment for donation feature

This is a Analytics & Experimentation interview question from PayPal for Data Scientist roles. View the full question and solution on PracHub.

Q: How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

Question

Product/Experimentation Case

Uber Eats is considering a new feature: when a user places an order, they can optionally donate (tip-like or charitable donation) to the merchant.

You are asked to propose:

1) Goal and hypotheses

What is the main product goal of this feature?
State primary and secondary hypotheses you would test.

2) Metrics selection

Define:

Primary (success) metric(s)
Diagnostic metrics (to understand mechanisms)
Guardrail metrics (to ensure no harm)

Be explicit about metric definitions (unit of analysis, numerator/denominator, time window).

3) Experiment design

Describe an experiment plan, including:

Experiment unit (user/order/restaurant)
Treatment(s) and control
Eligibility/targeting
How you handle repeated orders, interference/spillover, and novelty effects
How you would evaluate heterogeneous effects (e.g., by restaurant type, user frequency)

4) Risks and confounders

List major risks/biases (e.g., selection, cannibalization) and how you would mitigate them.

Deliverable

Produce a concise plan that an analytics/DS team could execute.