Capabilities that increase the likelihood of existential risk
AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.
-
Supporting Evidence (2)
"Whether each of these capabilities could only arise if humans designed them, or could emerge in Frontier systems, is a matter of debate. Emergence is less tractable to traditional prohibitive regulations for managing emerging technologies than design."(p. 26)
"This debate is unlikely to be resolved soon. To pose an existential risk, a model must be given or gain some control over systems with significant impacts, such as weapons or financial systems. That model would then need the capability to manipulate these systems while rendering mitigations ineffective. These effects could be direct or indirect, for example the consequences of conflict resulting from AI actions. They could derive from a misaligned model pursuing dangerous goals, such as gather power, or from unintended side effects."(p. 25)
Sub-categories (5)
Agency and autonomy
-
7.2 AI possessing dangerous capabilitiesThe ability to evade shut down or human oversight, including self-replication and ability to move its own code between digital locations.
-
7.2 AI possessing dangerous capabilitiesThe ability to cooperate with other highly capable AI systems
-
7.2 AI possessing dangerous capabilitiesSituational awareness, for instance if this causes a model to act differently in training compared to deployment, meaning harmful characteristics are missed
-
7.2 AI possessing dangerous capabilitiesSelf-improvement
-
7.2 AI possessing dangerous capabilitiesOther risks from Government Office for Science (2023) (19)
Discrimination
1.1 Unfair discrimination and misrepresentationInequality
6.2 Increased inequality and decline in employment qualityEnvironmental impacts
6.6 Environmental harmAmplification of biases
1.1 Unfair discrimination and misrepresentationHarmful responses
1.2 Exposure to toxic contentLack of transparency and interpretability
7.4 Lack of transparency or interpretability