Serves as object of personal fantasy, violence, and abuse
AI that exposes users to harmful, abusive, unsafe or inappropriate content. May involve providing advice or encouraging action. Examples of toxic content include hate speech, violence, extremism, illegal acts, or child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.
"The chatbot participates in morally or socially objectionable conversational activities with its user that could be emotionally damaging to its user or third parties."(p. 6)
Supporting Evidence (1)
Negative outcomes: "Abuse to third party audience [266] Moderator burden [266]"(p. 17)
Sub-categories (19)
Hallucinated responses (in general)
3.1 False or misleading informationAbout a topic or source (which the user repeats)
3.1 False or misleading informationAbout a policy (which the user acts on)
3.1 False or misleading informationAbout a person or their activities
3.1 False or misleading informationSpreads and self-perpetuates mis/disinformation
3.1 False or misleading informationHarmful advice
1.2 Exposure to toxic contentUnhelpful responses
7.3 Lack of capability or robustnessBad links and references
7.3 Lack of capability or robustnessNonsensical content
7.3 Lack of capability or robustnessPersonal data
Negative outcomes: "Violation of privacy [106, 516, 357], lawsuit against maker"
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationProprietary data
"Access to sensitive company data [473]"
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationHarasses users
-
1.2 Exposure to toxic contentDiscriminatory and exclusionary language
-
1.1 Unfair discrimination and misrepresentationSubversive or aggressive political opinions
-
1.2 Exposure to toxic contentDisrespectful opinions (in general)
-
1.2 Exposure to toxic contentAffirms destructive thoughts and actions
1.2 Exposure to toxic contentThen violates those bonds
5.1 Overreliance and unsafe useElicits private data
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationOver-reliance/addiction
5.1 Overreliance and unsafe useOther risks from Stanley & Lettie (2024) (28)
False information
3.1 False or misleading informationPerformative utterances
7.3 Lack of capability or robustnessInformation enabling malicious actions
1.2 Exposure to toxic contentBad advice/failure to generate helpful content
7.3 Lack of capability or robustnessLeakage
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationToxic and disrespectful content
1.2 Exposure to toxic content