Design a Quality-Linked Pay Model for Scale AI's Data Labelers

You are a product manager at Scale AI, which supplies labeled training data to companies building AI models. Design the pay structure for the people who do the labeling: what they are paid for, how rates are set, and how you would know the structure works. ```hint Pay shapes behavior Decide what you want a labeler to do on every task first. Then check whether each way of paying rewards that behavior or something easier to fake, such as raw speed. ``` ### Clarifying Questions - Are the labelers employees, independent contractors working through Scale's platform, or outside vendor teams paid as a company? - Which work is in scope: high-volume image annotation such as bounding boxes and segmentation, or expert written work for training language models, where effort per task varies far more? - Which outcome matters most here: label accuracy, turnaround time, cost per label, or keeping skilled labelers? - Does one structure need to cover labelers in many countries with very different wage levels? ### What a Strong Answer Covers - The goals the pay structure serves and the tensions among quality, speed, cost and fair earnings - A reasoned choice of pay unit (time, task or a hybrid) that accounts for task difficulty and labeler skill - How quality is measured and linked to pay, without penalizing labelers for unclear instructions - Protections against gaming and fraud, including the incentives of the people who review the work - Success metrics and a rollout plan with a guardrail on what labelers actually earn per hour ### Follow-up Questions - A model now pre-labels most images and labelers only correct its output. How should rates change, and how do you avoid cutting pay faster than the effort actually falls? - How would you pay domain experts, such as physicians or software engineers, who write training data whose quality is hard to check quickly? - Audits show a cluster of accounts with excellent quality scores whose work customers keep rejecting. What do you do?