Threats and Extortion

Sub-category

Risk Domain

7AI System Safety, Failures & Limitations

7.6Multi-agent risks

Risks from multi-agent interactions, due to incentives (which can lead to conflict or collusion) and/or the structure of multi-agent systems, which can create cascading failures, selection pressures, new security vulnerabilities, and a lack of shared information and trust.

"Threats and Extortion. A natural solution to problems of trust is to provide some kind of com- mitment ability to AI agents, which can be used to bind them to more cooperative courses of action. Unfortunately, the ability to make credible commitments may come with the ability to make credible threats, which facilitate extortion and could incentivize brinkmanship (see Section 2.2)."(p. 34)

Entity— Who or what caused the harm

Human

Due to a decision or action made by humans

AI system

Due to a decision or action made by an AI system

Other

Due to some other reason or is ambiguous

Intent— Whether the harm was intentional or accidental

Intentional

Due to an expected outcome from pursuing a goal

Unintentional

Due to an unexpected outcome from pursuing a goal

Other

Without clearly specifying the intentionality

Timing— Whether the risk is pre- or post-deployment

Pre-deployment

Occurring before the AI is deployed

Post-deployment

Occurring after the AI model has been trained and deployed

Other

Without a clearly specified time of occurrence

Supporting Evidence (1)

"For example, ransomware becomes more effective if the hacker can credibly commit to restore the victim’s data upon receiving payment, and coercion using AI-controlled weapons could become more frequent if actors gain the ability to make credible threats conditional on complicated demands (see also Case Study 11). More generally, an agent could use commitment devices to shift risks or costs to others, allowing it to behave irresponsibly.43 In other cases, it might be the agent that commits to an inflexible (cooperative) course of action which can be exploited by others who can adapt their strategies to this commitment.44 On the other hand, if used carefully, the ability to commit generally strictly empowers the committing agent (Letchford et al., 2013; Stengel & Zamir, 2010)."(p. 34)

Part of Commitment and Trust