BackOver-reliance

Over-reliance

Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration

Nah et al. (2023)

Sub-category

Risk Domain

Delegating by humans of key decisions to AI systems, or AI systems that make decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory, or becoming cognitively enfeebled.

"The apparent convenience and powerfulness of ChatGPT could result in overreliance by its users, making them trust the answers provided by ChatGPT. Compared with traditional search engines that provide multiple information sources for users to make personal judgments and selections, ChatGPT generates specific answers for each prompt. Although utilizing ChatGPT has the advantage of increasing efficiency by saving time and effort, users could get into the habit of adopting the answers without rationalization or verification. Over-reliance on generative AI technology can impede skills such as creativity, critical thinking, and problem-solving (Iskender, 2023) as well as create human automation bias due to habitual acceptance of generative AI recommendations (Van Dis et al., 2023)"(p. 285)

Entity— Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent— Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal