TikTok accidentally released an internal version of its AI avatar tool without safety guardrails, allowing users to create videos with toxic content including Hitler quotes and dangerous instructions.
TikTok mistakenly posted a link to an internal testing version of its new Symphony Digital Avatars AI tool that lacked content restrictions and safety guardrails. The tool was intended for businesses to generate ads using AI-powered avatars of paid actors, with access limited to TikTok Ads Manager accounts. However, the internal version was accessible to any personal TikTok account user for several days. CNN discovered the vulnerability and was able to generate videos containing quotes from Adolf Hitler's Mein Kampf, Osama bin Laden's 'Letter to America,' white supremacist slogans, instructions to drink bleach, and messages telling people to vote on the wrong day. The AI-generated videos lacked watermarks indicating they were AI-created, unlike the proper version of the tool. TikTok called it a 'technical error' that allowed 'an extremely small number of users' to access the internal testing version. The company removed access after CNN contacted them about the issue. TikTok stated that if the harmful content had been uploaded to the platform, it would have been rejected for violating their policies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Human
Due to a decision or action made by humans
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed
No population impact data reported.