Keeping Harmful Content Off Instagram at the Point of Upload

You are a product manager at Instagram. How would you keep "bad" content from being uploaded to the platform? The question is about social media platforms in general; use Instagram as the concrete product. Explain what you would count as bad content, which kinds you would tackle first, what you would change in the product to keep them from being posted, and how you would measure whether the changes work. ```hint Bad is not one thing Content that is illegal, content that breaks the platform's rules, and content that is merely unwanted differ in how much harm they cause and in how reliably they can be spotted at the moment of upload. Sort them first; the right point of intervention depends on the category. ``` ### Clarifying Questions - Does "bad" cover only illegal and rule-breaking content, or also legal but unwanted material such as spam, misleading claims and graphic images? - Must content be stopped before it is published, or can the plan include fast action in the minutes after a post goes live, such as limiting its reach until it is reviewed? - Which surfaces are in scope: feed posts, Reels, Stories, comments, live video, direct messages? - When catching more harmful content means wrongly blocking some legitimate posts, which way should the default lean? ### What a Strong Answer Covers - A definition of bad content split into categories by severity, volume and how detectable each is at upload, with a reasoned choice of where to start - Defenses at several points of the upload path (account trust before posting, automated checks during upload, prompts to the poster, fast containment after publishing) rather than a single filter - An explicit stance on the trade-off between missed harm and wrongly blocked posts, including notice and appeals for creators - Metrics that measure how much harmful content people actually see, with guardrails for false positives, upload speed and legitimate posting - Instagram-specific difficulties: the volume of images and video, bad actors who adapt, differences in law across countries, and privacy in direct messages ### Follow-up Questions - Your upload checks wrongly block a well-known creator's post and the mistake becomes news. What do you change, and what do you keep? - How would your approach differ for live video, where nothing can be checked before viewers see it? - Spammers begin making small edits to banned images so they no longer match your checks. How do you respond?