Multi-agent interactions

Safety and emergent functionality in multi-agent interactions

Understanding how individual agent dispositions and capabilities scale into complex multi-agent dynamics, evaluating emergent functionalities (e.g., coordinated strategies), enhancing robustness of LLM agents to correlated failures stemming from foundationality, and applying insights from multi-agent RL research to LLM-based systems.

Detecting and preventing collusion and emergent collective behavior

Developing detection techniques (e.g., information-theoretic or interpretability-based) for collusion between AI agents, benchmarking and evaluating collusive tendencies, designing mitigation strategies such as oversight regimes, communication restrictions, and methods for steering agents, understanding conditions (e.g., agent similarity, communication channels, environment structure) that facilitate collusion, and understanding why and how general “super-agents” might develop from many narrow agents.

Multi-agent security

Assessing unique security risks that arise in multi-agent ecosystems, designing defenses (e.g., secure communication protocols, improved network architectures, information security), studying how multiple systems can circumvent safeguards, evaluating robustness of cooperation to adversarial attacks (e.g., if a small number of malicious agents can destabilize larger groups), evaluating how well agents can adversarially attack each other, and studying the impact of AI agent’s training dynamics on data generated by each other with respect to shared vulnerabilities/correlated failure modes.

Network effects and destabilizing dynamics in agent ecosystem

Understanding which network structures and interaction patterns lead to robust or fragile systems, monitoring and controlling dynamics and co-adaptation of networks of advanced agents, and identifying important security concerns in existing and future multi-agent application areas (e.g., finance, energy grids) and applying lessons from those areas to manage destabilizing forces.

Transparency, information asymmetries, and communication protocols

Studying how agent transparency (e.g., code access) or predictability of agents can influence cooperation or defection, scaling Bayesian persuasion and information design to complex multi-agent settings, developing secure information transmission methods between AI agents to promote cooperation,, examining how agent similarity and evidential reasoning about others affect ability and propensity to cooperate, and developing efficient algorithms for zero- or few-shot coordination in high-stakes scenarios.

Multi-agent metrics and evaluations

Distinguishing and measuring cooperative dispositions, understanding agents’ robustness against coercion or exploitation, quantifying traits like altruism or spite, assessing the impact of capability asymmetries between agents, examining how training processes and data sources influence cooperation, and developing dangerous capability evaluations for multi-agent systems.