4chan users exploited ElevenLabs' voice cloning AI technology to create deepfake audio recordings of celebrities including Emma Watson, Joe Rogan, and Ben Shapiro saying racist, transphobic, and violent content.
In January 2023, users on the message board 4chan used ElevenLabs' voice cloning AI technology to create deepfake audio recordings of celebrities making offensive statements. ElevenLabs is a British AI startup founded by former Google and Palantir engineers that offers voice synthesis and cloning services through a beta platform. The company can generate voice clones from recordings as short as one minute. Specific examples included a generated voice resembling Emma Watson reading sections of Mein Kampf, a Ben Shapiro-like voice making racist remarks about Alexandria Ocasio-Cortez, and a Rick Sanchez voice making violent threats about domestic abuse. The clips were uploaded to 4chan on Sunday and ranged from harmless to violent, transphobic, homophobic, and racist content. ElevenLabs acknowledged finding an 'increasing number of voice cloning misuse cases' and announced plans to implement additional safeguards including payment verification, ID verification, and manual review of cloning requests. The company stated it could trace generated audio back to specific users and would ban accounts violating their policies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
Human
Due to a decision or action made by humans
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed