Offensive Cyber Operations (General)
Using AI systems to develop cyber weapons (e.g., by coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or chemical, biological, radiological, nuclear, and high-yield explosives), or use weapons to cause mass harm.
"Offensive cyber operations are malicious attacks on computer systems and networks aimed at gaining unauthorized access to, manipulating, denying, disrupting, degrading, or destroying the target system. These attacks can target the system’s network, hardware, or software. Advanced AI assistants can be a double-edged sword in cybersecurity, benefiting both the defenders and the attackers. They can be used by cyber defenders to protect systems from malicious intruders by leveraging information trained on massive amounts of cyber-threat intelligence data, including vulnerabilities, attack patterns, and indications of compromise. Cyber defenders can use this information to enhance their threat intelligence capabilities by extracting insights faster and identifying emerging threats. Advanced cyber AI assistant tools can also be used to analyze large volumes of log files, system output, or network traffic data in the event of a cyber incident, and they can ask relevant questions that an analyst would typically ask. This allows defenders to speed up and automate the incident response process. Advanced AI assistants can also aid in secure coding practices by identifying common mistakes in code and assisting with fuzzing tools. However, advanced AI assistants can also be used by attackers as part of offensive cyber operations to exploit vulnerabilities in systems and networks. They can be used to automate attacks, identify and exploit weaknesses in security systems, and generate phishing emails and other social engineering attacks. Advanced AI assistants can also be misused to craft cyberattack payloads and malicious code snippets that can be compiled into executable malware files."(p. 71)
Part of Malicious Uses
Other risks from Gabriel et al. (2024) (69)
Capability failures
7.3 Lack of capability or robustnessCapability failures > Lack of capability for task
7.3 Lack of capability or robustnessCapability failures > Difficult to develop metrics for evaluating benefits or harms caused by AI assistants
6.5 Governance failureCapability failures > Safe exploration problem with widely deployed AI assistants
7.3 Lack of capability or robustnessGoal-related failures
7.1 AI pursuing its own goals in conflict with human goals or valuesGoal-related failures > Misaligned consequentialist reasoning
7.3 Lack of capability or robustness