How do I approach Analytics & Experimentation interview questions?

Analytics & Experimentation questions require understanding of core concepts and practice. PracHub provides solutions with explanations to help you master analytics & experimentation interviews.

What difficulty level is this interview question?

This is a medium difficulty Analytics & Experimentation question, commonly asked during Onsite rounds at Uber.

What role is this question designed for?

This question is commonly asked for Data Scientist candidates at Uber during technical interviews.

Evaluate Marketplace Changes | Uber Interview Question

Q: Evaluate Marketplace Changes

This question evaluates a data scientist's competency in experimental design, causal inference, metrics instrumentation, A/B testing, and marketplace analytics within the Analytics & Experimentation domain, focusing on both conceptual understanding of bias, interference, and decision framing and practical application of randomization, metric selection, and heterogeneity analysis. It is commonly asked to assess the ability to define clear decision goals, identify primary and guardrail metrics, determine the correct unit of randomization or analysis, understand when experiments versus quasi-experimental designs are appropriate, and anticipate sources of bias or interference across regions, drivers, merchants, and users.

You are a marketplace data scientist at a mobility and delivery platform. Discuss how you would evaluate the following product and algorithm changes:

Driver coupon program : the company offers coupons or incentives to drivers to increase supply. How would you measure whether the coupon creates incremental value rather than simply paying drivers who would have driven anyway?
Batched or bunch delivery : one driver can complete multiple deliveries in a single route. How would you evaluate whether this improves marketplace efficiency without hurting customer experience?
Split delivery fee and service fee : instead of showing a single total platform fee, the app now displays separate delivery and service fees. How would you test whether this pricing change affects conversion, trust, and unit economics? Then explain how you would specifically deep dive on the impact of the delivery fee level.
Reduced ETA model computation time : the ETA estimation system becomes faster to compute. How would you evaluate the business impact of lower latency, and what randomization unit would you choose?

For each case, specify:

the decision goal
primary metrics and guardrail metrics
the correct unit of randomization or analysis
key sources of bias or interference
when you would use an experiment versus a quasi-experimental design
how you would handle heterogeneity across regions, drivers, merchants, or users

You are a marketplace data scientist at a mobility and delivery platform. Discuss how you would evaluate the following product and algorithm changes:

Driver coupon program : the company offers coupons or incentives to drivers to increase supply. How would you measure whether the coupon creates incremental value rather than simply paying drivers who would have driven anyway?
Batched or bunch delivery : one driver can complete multiple deliveries in a single route. How would you evaluate whether this improves marketplace efficiency without hurting customer experience?
Split delivery fee and service fee : instead of showing a single total platform fee, the app now displays separate delivery and service fees. How would you test whether this pricing change affects conversion, trust, and unit economics? Then explain how you would specifically deep dive on the impact of the delivery fee level.
Reduced ETA model computation time : the ETA estimation system becomes faster to compute. How would you evaluate the business impact of lower latency, and what randomization unit would you choose?

For each case, specify:

the decision goal
primary metrics and guardrail metrics
the correct unit of randomization or analysis
key sources of bias or interference
when you would use an experiment versus a quasi-experimental design
how you would handle heterogeneity across regions, drivers, merchants, or users

Evaluate Marketplace Changes

Quick Overview

Solution

Submit Your Answer

Evaluate Marketplace Changes

Quick Overview

Solution

Submit Your Answer