Impacts of AI (Cyberattacks)

Automated discovery and exploitation of software systems

"GPAIs can be used to aid in the automated discovery of software vulnerabilities [33]. This can empower malicious actors, making their cyberattacks more effi- cient and potentially more damaging. This type of automation allows attackers to expand the scale of their operations at a low cost, increasing the impact of their actions. New malware can be developed automatically, or the known vulnerabilities can be exploited to create more sophisticated attacks."

Amplification of cyberattacks

"General-purpose AI models may significantly enhance the magnitude and ef- fectiveness of cyberattacks, by amplifying existing capabilities or resources of malicious actors [3]. For example, GPAI models may be employed to: • Automatically scan open-source codebases and compiled binaries for po- tential vulnerabilities • Apply known exploits flexibly and at scale (e.g., identifying vulnerable computers based on subtle cues in response times or output formats) • Assist with different aspects of cyberattacks, including planning, recon- naissance, exploit searching, remote control, malware implementation, and data exfiltration • Combine social engineering (phishing, deepfakes, etc.) with cyberattacks at scale."

AI-driven spear phishing attacks

"Generative models can be misused to target individual users more efficiently by using personalized information [23]. Highly convincing automated fraudulent schemes can exploit the trust of victims by extracting sensitive data and making the deception more likely to succeed. For example, in LLMs, this misuse can be aided by jailbreaking techniques [178]."

Models generating code with security vulnerabilities

"Models can generate code or coding suggestions that contain security vulner- abilities. This may occur across various LLM-based model families, including more advanced models with superior coding performance, where the tendency to produce insecure code is even more pronounced [26]."