Analyze Cancellation Change with Statistics

Quick Overview

This question evaluates statistical inference for binary outcomes, regression adjustment for confounding, experiment design and power analysis, and nonparametric uncertainty estimation using bootstrapping, and is targeted at Data Scientist roles within the Statistics & Math domain.

Analyze Cancellation Change with Statistics

Company: Uber

Role: Data Scientist

Category: Statistics & Math

Difficulty: medium

Interview Round: Technical Screen

Uber introduces a small product tweak to reduce cancellations. You observe: - Before: n1 = 50,000 requested trips, cancel rate p1 = 7.2%. - After: n2 = 48,000 requested trips, cancel rate p2 = 6.6%. Assume independent Bernoulli outcomes per request to start. Answer: 1) Choose an appropriate hypothesis test for the change in cancellation rate and compute a 95% CI for (p_after − p_before). State H0/HA and interpret the CI practically. 2) Now consider confounding by city and hour-of-day. Specify a logistic regression that adjusts for these (and an interaction term you deem important). Write the model formula, list key assumptions, and describe how you would check calibration and overdispersion. 3) Power/MDE: What per-arm sample size would you need to detect an absolute change of 0.5 percentage points at α=0.05 with 80% power (ignore clustering first)? Then discuss how an intraclass correlation of 0.02 at the driver level would inflate the required N under a cluster-robust design; show the design effect and the new N. 4) Nonparametric check: Describe a bootstrap procedure to form a CI for the rate difference stratified by city (stratified resampling) and when its CI might be preferable to the z-approximation.

Quick Answer: This question evaluates statistical inference for binary outcomes, regression adjustment for confounding, experiment design and power analysis, and nonparametric uncertainty estimation using bootstrapping, and is targeted at Data Scientist roles within the Statistics & Math domain.

|Home/Statistics & Math/Uber
Uber logo
Uber
Oct 13, 2025, 9:49 PM
mediumData ScientistTechnical ScreenStatistics & Math
6
0

A/B change in cancellation rate (before vs after)

Context: You are evaluating a small product tweak intended to reduce cancellations. Treat each trip request as a Bernoulli outcome (1 = canceled, 0 = not canceled) to start.

Observed data:

  • Before: n1 = 50,000 requested trips, cancel rate p1 = 7.2%.
  • After: n2 = 48,000 requested trips, cancel rate p2 = 6.6%.

Tasks:

  1. Hypothesis test and CI
    • Choose an appropriate test for the change in cancellation rate. State H0 and HA.
    • Compute a 95% confidence interval for the absolute rate difference Δ = (p_after − p_before) and interpret it in practical terms.
  2. Adjustment for confounding (city, hour-of-day)
    • Specify a logistic regression that adjusts for city and hour-of-day, and include one interaction you deem important.
    • Write the model formula, list key assumptions, and describe how you would check calibration and overdispersion.
  3. Power and MDE
    • What per-arm sample size is required to detect an absolute change of 0.5 percentage points at α = 0.05 with 80% power (ignore clustering first)?
    • Then discuss how an intraclass correlation (ICC) of 0.02 at the driver level inflates the required N under a cluster-robust design. Show the design effect and the new N.
  4. Nonparametric check (bootstrap)
    • Describe a bootstrap procedure to form a CI for the rate difference, stratified by city (stratified resampling).
    • Explain when this CI might be preferable to the z-approximation.
Loading comments...