Websites used AI to create and distribute nonconsensual deepfake nude images of women and girls, leading to a successful lawsuit by San Francisco City Attorney's Office that shut down 11 websites.
The San Francisco City Attorney's Office filed a first-of-its-kind lawsuit in August 2024 against 16 websites that create and distribute nonconsensual AI-generated pornography. These websites used sophisticated AI models trained on pornographic images to generate realistic nude images of real, identifiable women and girls without their consent. The websites offered user-friendly interfaces for uploading clothed images to generate pornographic versions, requiring users to subscribe or pay for the service. Collectively, these websites received over 200 million visits in the first six months of 2024. Some websites specifically promoted the nonconsensual nature, with one stating 'Imagine wasting time taking her out on dates, when you can just use [website] to get her nudes.' The lawsuit alleged violations of state and federal laws prohibiting deepfake pornography, revenge pornography, and child pornography. As a result of the legal action filed over six months ago, 11 of the targeted websites have shut down permanently. The case highlighted how AI-generated nonconsensual explicit images cause immeasurable harm to victims including extortion, bullying, threats, and humiliation, with examples ranging from Taylor Swift to middle school students.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
AI system
Due to a decision or action made by an AI system
Intentional
Due to an expected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed