Diagnose drop in shopper order acceptance

Quick Overview

A PayPal data-scientist onsite analytics case: shopper order acceptance in a grocery-delivery marketplace drops by two-thirds on Sunday afternoon, and you must diagnose the cause. It tests precise metric definition and funnel decomposition, supply/demand/merchant root-cause hypotheses, metric-driven investigation, ruling out logging artifacts, and proposing mitigations plus a switchback experiment.

Diagnose drop in shopper order acceptance

Company: PayPal

Role: Data Scientist

Category: Analytics & Experimentation

Difficulty: easy

Interview Round: Onsite

##### Question Marketplace diagnosis case. A grocery-delivery marketplace (Instacart-style) observes that on **Sunday afternoon**, the number of orders that shoppers **accept** drops by about **2/3** compared to the usual baseline. Assume this is a same-day change (not a long multi-month trend). You have access to typical marketplace logs: order creation / checkout, dispatch and offer events, acceptances, cancellations, ETAs, shopper app events, and merchant/store signals. Acting as the on-call, bar-raiser-style interviewer, diagnose the problem. 1. **Clarify the metric.** Define precisely what "orders accepted" means and which denominator(s) matter (e.g. accepted count vs. acceptance rate = accepted / offered). State a clean funnel decomposition. 2. **Build a structured root-cause tree** from three perspectives: (a) **shopper supply / behavior**, (b) **customer demand / order mix**, and (c) **merchant / store operations and platform systems**. 3. **List the key metrics and slices you would check first** (at least 10), and state what pattern in each would support or refute a hypothesis. 4. **Distinguish a real behavioral change from a logging / measurement artifact** — how would you confirm the drop is real? 5. **Propose immediate mitigations and longer-term fixes**, including at least one experiment or controlled rollout plan to validate the leading hypothesis and prevent recurrence, and state the "next action" you would recommend.

Quick Answer: A PayPal data-scientist onsite analytics case: shopper order acceptance in a grocery-delivery marketplace drops by two-thirds on Sunday afternoon, and you must diagnose the cause. It tests precise metric definition and funnel decomposition, supply/demand/merchant root-cause hypotheses, metric-driven investigation, ruling out logging artifacts, and proposing mitigations plus a switchback experiment.

|Home/Analytics & Experimentation/PayPal
PayPal logo
PayPal
Oct 10, 2025, 12:00 AM
easyData ScientistOnsiteAnalytics & Experimentation
4
0
Question

Marketplace diagnosis case. A grocery-delivery marketplace (Instacart-style) observes that on Sunday afternoon, the number of orders that shoppers accept drops by about 2/3 compared to the usual baseline.

Assume this is a same-day change (not a long multi-month trend). You have access to typical marketplace logs: order creation / checkout, dispatch and offer events, acceptances, cancellations, ETAs, shopper app events, and merchant/store signals. Acting as the on-call, bar-raiser-style interviewer, diagnose the problem.

  1. Clarify the metric. Define precisely what "orders accepted" means and which denominator(s) matter (e.g. accepted count vs. acceptance rate = accepted / offered). State a clean funnel decomposition.
  2. Build a structured root-cause tree from three perspectives: (a) shopper supply / behavior , (b) customer demand / order mix , and (c) merchant / store operations and platform systems .
  3. List the key metrics and slices you would check first (at least 10), and state what pattern in each would support or refute a hypothesis.
  4. Distinguish a real behavioral change from a logging / measurement artifact — how would you confirm the drop is real?
  5. Propose immediate mitigations and longer-term fixes , including at least one experiment or controlled rollout plan to validate the leading hypothesis and prevent recurrence, and state the "next action" you would recommend.
Loading comments...