BackAutomation bias
Automation bias
Risk Domain
Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.
"The tendency for humans to over-rely on AI models and systems, trusting their outputs without sufficient critical evaluation, which can lead to poor decision-making."(p. 11)
Entity— Who or what caused the harm
Intent— Whether the harm was intentional or accidental
Timing— Whether the risk is pre- or post-deployment
Other risks from Uuk2025 (60)
Control
7.1 AI pursuing its own goals in conflict with human goals or valuesAI systemIntentionalPost-deployment
Democracy
6.0 Socioeconomic & EnvironmentalOtherOtherOther
Discrimination
1.1 Unfair discrimination and misrepresentationOtherOtherPost-deployment
Economy
6.2 Increased inequality and decline in employment qualityOtherOtherPost-deployment
Environment
6.6 Environmental harmAI systemUnintentionalPost-deployment
Governance
6.5 Governance failureAI systemUnintentionalPost-deployment