AI systems may develop or acquire capabilities that can cause large-scale harm if used by humans, misaligned AI systems, or due to a failure in the AI system. These capabilities are described as dangerous because they can be used to threaten security or exercise control over humans. These capabilities may be intentionally designed into an AI system, may emerge unpredictably during development or training of a system, may be acquired by an AI system in its environment (e.g., through the use of tools), or be provided by a user.
One example of a dangerous capability is manipulation and persuasion, where an AI system can convince humans to believe things that are irrational or false or to engage in dangerous behaviors. Other dangerous capabilities include political strategy and knowledge of social dynamics that can be used to obtain and wield power. Cyber-offense skills may enable an AI system to gain ongoing unauthorized access to hardware, software, or data systems and work strategically towards a planned goal while minimizing the risk of detection. AI systems could hack into control systems and military hardware, allowing it to commandeer weapons.
AI systems may also develop highly effective "evasion skills," such as situational awareness and deception, which would allow them to outmaneuver human oversight and control. AIs may also acquire a suite of capabilities necessary for self-proliferation, including skills to escape operational confines and evade detection, autonomously produce income, obtain server space or computational resources, and copy their underlying software and parameters.
The highest risk scenarios in this subcategory are likely to arise not from a single capability, but from the convergence of several capabilities. Each of these dangerous capabilities may be used by an AI system to cause harm when intentionally directed by human actors, or employed by a misaligned AI to deceive or manipulate humans, gain resources, and evade shutdown or control.
Excerpt from the MIT AI Risk Repository full report
AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.
Incident volume relative to governance coverage — each dot is one of 24 subdomains
Entity
Who or what caused the harm
Intent
Whether the harm was intentional or accidental
Timing
Whether the risk is pre- or post-deployment
No recorded incidents for this subdomain.
Risks may still apply even without documented incidents.
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
157 shared governance docs
Using AI systems to develop cyber weapons (e.g., by coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or chemical, biological, radiological, nuclear, and high-yield explosives), or use weapons to cause mass harm.
148 shared governance docs
AI developers or state-like actors competing in an AI ‘race’ by rapidly developing, deploying, and applying AI systems to maximize strategic or economic advantage, increasing the risk they release unsafe and error-prone systems.
136 shared governance docs
Inadequate regulatory frameworks and oversight mechanisms that fail to keep pace with AI development, leading to ineffective governance and the inability to manage AI risks appropriately.
127 shared governance docs
Establishes the Artificial Intelligence Futures Steering Committee by April 1, 2026, under the Secretary of Defense. Directs it to develop policies for AI adoption, assess AI trajectories, and analyze AI risks and adversary developments. Requires quarterly meetings and a report to U.S. Congress by January 31, 2027.
Requires the Secretary of Defense to develop a cybersecurity policy for AI/ML systems no later than 180 days after the act is passed. Develop a comprehensive review of the effectiveness of the AI/ML policies. Addresses potential security risks, implements methods to mitigate those risks, and establishes standard policy. Requires a comprehensive report of the threats and cybersecurity measures by August 31, 2026.
Instructs the Secretary of Defense to develop a cybersecurity framework for Department of Defense AI and machine learning technologies. Requires tailoring of security requirements in consideration of costs versus benefits and encourages collaboration with private sector and academia.