Success Metrics for a PayPal Transaction-Categorization and Spending-Insights API

Imagine you are a product manager at PayPal who owns an API that analyzes what users spend and sorts their transactions into spending categories. Which key metrics would you track to decide whether this feature is a success? Say which metric matters most and how you would measure each one. ```hint An API has two kinds of customers Ask who calls the API and who ultimately sees its categories. Whether teams adopt it, whether its categories are right, and whether people act on the resulting insights are different layers; decide which layer tells you the feature is really working. ``` ### Clarifying Questions - Is the API used only by PayPal's own consumer and merchant products, or is it also offered to outside developers and partners? - Does it categorize each transaction in real time as it happens, or in batches for periodic summaries? - What do users get from the categories: a spending breakdown, budgets and alerts, or input to other features such as offers? - Is the main goal more engagement and retention in PayPal's products, or revenue from the API itself? ### What a Strong Answer Covers - A stated goal for the feature and its users: the teams or developers who integrate it and the people whose spending is categorized. - One north-star metric that reflects value delivered, with a reason it beats raw call volume. - Supporting metrics across adoption, categorization quality, performance and reliability, and end-user engagement, each with how it would be measured. - Guardrails covering privacy and trust, cost, latency and errors in sensitive spending categories. - How the metrics would be validated, such as a labeled evaluation set and a holdout experiment, and the trade-offs they expose, such as coverage versus accuracy. ### Follow-up Questions - Users who see spending categories retain better than users who do not. How would you show the feature causes that difference? - Overall accuracy is high, but users keep re-labeling transactions in one category. How would you investigate? - If you had to cut the dashboard to three metrics for a quarterly review, which would you keep?