Design metrics for a short-video app's direct messaging feature

A short-video social app has a direct messaging (DM) feature: users can send private messages to one another and share videos from their feed into a conversation. How would you design the metrics for this feature? Say what you would measure to know whether it is succeeding, which metric you would treat as the north star, and which guardrails you would watch. The report gives only the prompt, designing metrics for the direct message feature; the description of the feature above is an assumption to confirm with the interviewer. ```hint Value on both sides A sent message creates value only if someone receives it and engages. Think about what separates a conversation from a broadcast or from spam. ``` ```hint The rest of the app Time spent in the inbox is time not spent watching videos. Ask what a DM metric could hide about the main product. ``` ### Clarifying Questions - What is the company's goal for DM: deeper connections between users, more sharing of videos, retention of the whole app, or a mix? Which comes first? - Is DM newly launched or mature? That changes whether adoption or depth matters more. - Who can message whom: only mutual followers, or anyone, subject to privacy settings? Are group chats included? - Will the metrics feed a health dashboard, experiment readouts, or both? - Are some users, such as younger users, restricted from messaging, and should they be excluded from the denominators? ### What a Strong Answer Covers - The feature's goal stated before any metric, and a north star tied to that goal with a precise definition - Supporting metrics across adoption, engagement depth, reciprocity, video sharing and retention, each with a numerator, denominator and time window - Guardrails for spam, harassment and safety, and for cannibalization of video viewing - How each metric could mislead or be gamed, and the segments to cut by - How the metrics serve dashboards and experiments, including interference between senders and recipients ### Follow-up Questions - An experiment that makes it easier to share a video into a DM raises DMs sent per user but slightly lowers watch time per user. How do you decide whether to launch? - The north star jumps overnight in one market. How do you tell real growth from spam or a logging change? - Why does randomizing individual users give a biased estimate in a DM experiment, and how would you design around it? - Users who use DM retain better than users who do not. How would you find out whether DM causes the higher retention?