Unpredictable outcomes
AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
"Our culture, lifestyle, and even probability of survival may change drastically. Because the intentions programmed into an artificial agent cannot be guaranteed to lead to a positive outcome, Machine Ethics becomes a topic that may not produce guaranteed results, and Safety Engineering may correspondingly degrade our ability to utilize the technology fully."(p. 688)
Other risks from Meek et al. (2016) (17)
Unethical decision making
7.3 Lack of capability or robustnessPrivacy
2.1 Compromise of privacy by leaking or correctly inferring sensitive informationHuman dignity/respect
5.2 Loss of human agency and autonomyDecision making transparency
7.4 Lack of transparency or interpretabilitySafety
7.3 Lack of capability or robustnessLaw abiding
7.3 Lack of capability or robustness