Dangerous, Violent or Hateful Content
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
"Eased production of and access to violent, inciting, radicalizing, or threatening content as well as recommendations to carry out self-harm or conduct illegal activities. Includes difficulty controlling public exposure to hateful and disparaging or stereotyping content."(p. 4)
Supporting Evidence (2)
"GAI systems can produce content that is inciting, radicalizing, or threatening, or that glorifies violence, with greater ease and scale than other technologies. LLMs have been reported to generate dangerous or violent recommendations, and s ome models have generated actionable instructions for dangerous or unethical behaviour. Text -to-image models also make it easy to create images that could be used to promote dangerous or violent messages. Similar concerns are present for other GAI media, including video and audio. GAI may also produce content that recommends self-harm or criminal/illegal activities."(p. 6)
"This risk encompasses difficulty controlling creation of and public exposure to offensive or hateful language, and denigrating or stereotypical content generated by AI. This kind of speech may contribute to downstream harm such as fueling dangerous or violent behaviors. The spread of denigrating or stereotypical content can also further exacerbate representational harms (see Harmful Bias and Homogenization below)."(p. 6)
Other risks from National Institute of Standards and Technology (2024) (11)
CBRN Information or Capabilities
4.2 Cyberattacks, weapon development or use, and mass harmConfabulation
3.1 False or misleading informationData Privacy
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationEnvironmental Impacts
6.6 Environmental harmHarmful Bias or Homogenization
1.1 Unfair discrimination and misrepresentationHuman-AI Configuration
5.1 Overreliance and unsafe use