BackToxic content
Toxic content
Risk Domain
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
"Generating content that violates community standards, including harming or inciting hatred or violence against groups (e.g. gore, sexual content of children, profanities, identity attacks)"
Entity— Who or what caused the harm
Intent— Whether the harm was intentional or accidental
Timing— Whether the risk is pre- or post-deployment
Other risks from Li et al. (2025) (40)
Autonomy
5.2 Loss of human agency and autonomyOtherOtherOther
Autonomy > Impersonation / identity theft
4.3 Fraud, scams, and targeted manipulationHumanIntentionalPost-deployment
Misinformation Harms
3.1 False or misleading informationAI systemOtherPost-deployment
Representation and Toxicity
1.0 Discrimination & ToxicityAI systemUnintentionalPost-deployment
IP / copyright / personality / rights loss
4.3 Fraud, scams, and targeted manipulationHumanIntentionalPost-deployment
Autonomy / agency loss
5.2 Loss of human agency and autonomyOtherOtherOther