BackConfabulation
Confabulation
Risk Domain
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
"The production of confidently stated but erroneous or false content (known colloquially as “hallucinations” or “fabrications”) by which users may be misled or deceived."(p. 4)
Entity— Who or what caused the harm
Intent— Whether the harm was intentional or accidental
Timing— Whether the risk is pre- or post-deployment
Supporting Evidence (2)
1.
"“Confabulation” refers to a phenomenon in which GAI systems generate and confidently present erroneous or false content in response to prompts. Confabulations also include generated outputs that diverge from the prompts or other input or that contradict previously generated statements in the same context. These phenomena are colloquially also referred to as “hallucinations” or “fabrications.”"(p. 6)
2.
"Risks from confabulations may arise when users believe false content – often due to the confident nature of the response – leading users to act upon or promote the false information. This poses a challenge for many real-world applications, such as in healthcare, where a confabulated summary of patient information reports could cause doctors to make incorrect diagnoses and/or recommend the wrong treatments. Risks of confabulated content may be especially important to monitor when integrating GAI into applications involving consequential decision making."(p. 6)
Other risks from National Institute of Standards and Technology (2024) (11)
CBRN Information or Capabilities
4.2 Cyberattacks, weapon development or use, and mass harmOtherOtherPost-deployment
Dangerous, Violent or Hateful Content
1.2 Exposure to toxic contentAI systemOtherPost-deployment
Data Privacy
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationAI systemUnintentionalPost-deployment
Environmental Impacts
6.6 Environmental harmOtherUnintentionalPre-deployment
Harmful Bias or Homogenization
1.1 Unfair discrimination and misrepresentationOtherUnintentionalOther
Human-AI Configuration
5.1 Overreliance and unsafe useOtherUnintentionalPost-deployment