Handling Duplicate Upstream Order Records in Dasher Pay Calculation

A delivery platform pays its Dashers from order activity. During every minute, a Dasher earns a base rate multiplied by the number of orders they have in progress, and the pay is doubled inside designated double-pay windows. The calculation learns when each order was accepted and when it was delivered from records produced by an upstream system. The upstream data can contain duplicates. How do you handle them so that every Dasher is paid correctly? ```hint What should a repeat change? Picture the same delivery record being processed twice. Decide what must be identical after the second pass, and what key lets you recognize the record as something you have already seen. ``` ```hint Not every repeat is a copy Two records about the same order can disagree. Decide whether they describe the same fact, and how you choose between them. ``` ### Constraints and Clarifications - This came up as a system-design follow-up to coding the pay calculation; you do not need to write the calculation again. - Not given: the volume of records, whether pay is computed continuously or once per pay period, and whether records carry a unique ID. Ask, or state your assumptions. ### Clarifying Questions - What does a duplicate look like: the same record delivered twice with identical content, or two records for the same order and action with different timestamps? - Does each record carry a unique ID assigned by the producer, or only the order, the action and a timestamp? - Is pay computed incrementally as records arrive, or recomputed for a pay period from all of its records? - Can records arrive late or out of order, and can an order's data be corrected after the Dasher has been paid? ### What a Strong Answer Covers - Defines duplicates precisely, separating exact replays from re-emitted facts and conflicting versions, and picks a key that identifies one fact - Shows concretely how a duplicate distorts pay, and where in the pipeline the design neutralizes repeated records - Explains why an at-least-once pipeline produces duplicates even with a careful producer, and what messaging-level exactly-once guarantees do not cover - Handles conflicting and late records, corrections after a period is paid, and repeated payout attempts - Bounds the cost of deduplication state and adds metrics that expose duplicate and conflict rates ### Follow-up Questions - A corrected delivery time arrives after the Dasher has already been paid for that period. What happens? - Your deduplication data grows without bound. How do you limit it, and what risk does that create? - The service that sends payouts times out and retries. How do you make sure a Dasher is not paid twice?