Product plan for catching and limiting deepfake misinformation on TikTok

Deepfake videos keep getting more convincing. As a product manager at TikTok, how would you use AI to detect deepfakes used for misinformation and stop them from misleading people on the platform? Explain which harms you would prioritize, how detection would work, what the product should do once a video is flagged, and how you would know the approach is working. ```hint Not every deepfake is misinformation Much synthetic video is harmless: effects, filters, parody, clearly labeled AI art. Decide which kinds of synthetic content actually deceive people in harmful ways before choosing what to detect, so the system targets harm without punishing creativity. ``` ### Clarifying Questions - Is the scope deceptive synthetic video only, or misinformation in general with deepfakes as one source of it? - Which harms matter most: election and civic misinformation, fabricated news events, impersonation of real people for scams, or non-consensual content? - Should videos made with TikTok's own AI effects be treated differently from videos created with outside tools and uploaded? - Should the answer focus on the detection product, or also cover enforcement policy and creator rules? ### What a Strong Answer Covers - A harm-based scope that separates deceptive, harmful deepfakes from creative, satirical or labeled synthetic content - A layered detection design (provenance and disclosure, model-based detection, account and spread signals, human review) and its limits as generators improve - Graduated responses tied to harm and confidence, such as labels, reduced distribution, friction before sharing, removal and account penalties - Speed: acting before a video is amplified by the recommendation feed - Metrics for detection quality and user exposure, with guardrails for false positives and an appeals path for creators ### Follow-up Questions - A video of a politician is flagged with medium confidence days before an election while it is spreading fast. What does the product do? - How would you keep detection working as generation tools improve and bad actors adapt? - How would you handle creators whose real footage is wrongly labeled as a deepfake?