Measuring success for a robot Meta is building: what to track

Facebook (now Meta) is designing a robot. What metrics would you consider as part of that design work? ```hint Pin down the robot first Metrics only mean something against a job, so settle what this robot is for and who uses it before listing anything to measure. ``` ### Clarifying Questions - Is this a physical robot, or a software bot that people talk to inside Messenger or another Meta app? - What job is it meant to do, and for whom: people at home, businesses, or Meta's own operations such as its data centers? - Do you want metrics that guide design decisions during testing, success metrics after launch, or both? ### What a Strong Answer Covers - A stated assumption about the robot's purpose and first users, with a reason, and a goal metric tied to that purpose - Measures of whether the robot does its job well, kept separate from measures of whether people use it - Safety, privacy and trust measures suited to a robot built by Meta - Reliability and cost measures that decide whether it can ship and scale - Prioritization among the metrics, the trade-offs between them, and which apply in testing versus after launch ### Follow-up Questions - Usage is high, but most sessions end with a person stepping in to finish the task. What does that tell you, and what would you change? - How would your metrics change if the robot were the other kind: a chatbot instead of a physical device, or the reverse? - Which single metric would block a launch no matter how good the others look?