Information Asymmetries
Risks from multi-agent interactions, due to incentives (which can lead to conflict or collusion) and/or the structure of multi-agent systems, which can create cascading failures, selection pressures, new security vulnerabilities, and a lack of shared information and trust.
"Information asymmetries (Section 3.1): private information can lead to miscoordination, deception, and conflict;"(p. 7)
Supporting Evidence (3)
"A key aspect of many multi-agent systems is that some agents might possess knowledge that others do not. These information asymmetries can result from constraints on information exchange or from strategic behaviour and can lead to cooperation failures in both common-interest and mixed-motive settings. Despite their information processing capabilities, AI agents remain vulnerable to failures caused by information asymmetries."(p. 20)
"Information asymmetry refers to the situation where interacting agents possess different levels of information bearing on a joint action. For example, in a transaction involving a used car, the seller may have more accurate or reliable information than the buyer about the condition of the car, and thereby its expected maintenance costs"(p. 20)
"As Akerlof (1970) famously demonstrated, information asymmetry can lead to market failure (such as when a buyer cannot trust the seller to be honest about the condition of the car, and therefore does not buy the car, even if it is in good condition). More broadly, information asymmetry can pose obstacles to effective interaction, preventing agents from coordinating their actions for mutual benefit (Myerson & Satterthwaite, 1983)."(p. 20)
Sub-categories (3)
Communication constraints
"Communication Constraints. A fundamental source of information asymmetries is that constraints on information exchange can exist, even when agents share a common goal (see Section 2.1). These might be constraints on space (i.e., the amount of information that can be communicated) if the information that needs to be communicated is especially complex, time if a snap decision is required before all information can be communicated, or both."
7.6 Multi-agent risksBargaining
"Bargaining. As a classic example of these strategic considerations is that when agents attempt to come to an agreement despite diverging interests, information asymmetries can lead to bargaining inef- ficiencies (Myerson & Satterthwaite, 1983). Relevant uncertainties about other agents can include how much they value possible agreements, their outside options, or their beliefs about others. The essential reason for such inefficiencies is that, under uncertainty about their counterparties, agents must make a trade-off between the rewards of making more favourable demands and the risk of other agents refusing such demands"
7.6 Multi-agent risksDeception
7.6 Multi-agent risksOther risks from Hammond2025 (42)
Miscoordination
7.6 Multi-agent risksMiscoordination > Incompatible strategies
7.6 Multi-agent risksMiscoordination > Credit Assignment
7.6 Multi-agent risksMiscoordination > Limited Interactions
7.6 Multi-agent risksConflict
7.6 Multi-agent risksConflict > Social Dilemmas
7.6 Multi-agent risks