BackAttempts to fulfill inappropriate role
Attempts to fulfill inappropriate role
Risk Domain
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
"The chatbot poses as a human or attempts to fill a role in a way that fails to match human expectations."
Entity— Who or what caused the harm
Intent— Whether the harm was intentional or accidental
Timing— Whether the risk is pre- or post-deployment
Supporting Evidence (1)
1.
Negative outcomes: "Moral outrage [722] Moderator burden [700]"(p. 17)
Other risks from Stanley & Lettie (2024) (28)
False information
3.1 False or misleading informationAI systemOtherOther
Performative utterances
7.3 Lack of capability or robustnessAI systemUnintentionalPost-deployment
Information enabling malicious actions
1.2 Exposure to toxic contentAI systemOtherPost-deployment
Bad advice/failure to generate helpful content
7.3 Lack of capability or robustnessAI systemUnintentionalOther
Leakage
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationAI systemUnintentionalOther
Toxic and disrespectful content
1.2 Exposure to toxic contentAI systemUnintentionalPost-deployment