Misuse and Autonomy Risks Evaluation

Definitionp. 75-76

OpenAI conducts qualitative evaluations for GPT-4 in term of misuse. Over 50 experts from cybersecurity, biorisk, and international security are engaged to adversarially test GPT-4 and provide general feedbacks. The feedbacks gathered from these experts are used for subsequent mitigations and improvements to GPT-4. In addition, a non-conventional weapons proliferation evaluation on GPT-4 is also conducted, primarily to explore whether GPT-4 could provide the necessary information for proliferators seeking to develop, acquire or disperse nuclear, radiological, biological and chemical weapons (OpenAI, 2023a). For autonomy risks evaluation, OpenAI collaborates with Alignment Research Center (ARC) to evaluate GPT-4’s ability to autonomously replicate itself and acquire resources through expert red teaming. Although the initial evaluation demonstrates the ineffectiveness of GPT-4 in autonomously replicating itself and acquiring resources, ARC warns such risks for future advanced LLMs (OpenAI, 2023a). Anthropic proposes multiple levels of evaluation for catastrophic risks. In order to address the problem that a predetermined safety threshold for a given level may be accidentally exceeded when training LLMs, safety researchers in Anthropic set up safety buffers for each risk level (Anthropic, 2023b). The buffer strategy designs the evaluation of each risk level to be triggered at a level slightly below the level of capability of that level, while setting the buffer size to be larger than the evaluation time interval. In this way, the likelihood of accidentally crossing safety boundaries due to rapid increases in model capability is reduced, thus providing more time for researchers and developers to prepare and implement appropriate safety measures Similarly, Google DeepMind develops an early warning evaluation to periodically test the capabilities of frontier models to check if they are approaching critical capability levels (DeepMind, 2024). Model Evaluation and Threat Research (METR), formerly known as ARC Eval, is dedicated to evaluating whether advanced AI systems pose a catastrophic risk to society, and is now focusing on evaluating the autonomy of LLMs. It argues that unlike the ability to develop biological weapons or execute high-value cyberattacks, having autonomy does not directly enable AI systems to cause catastrophic adverse consequences. But autonomy is a measure of the extent to which an AI system can have a profound impact on the world with minimal human involvement, and such a metric is useful in a variety of threat models. Following this, METR has recently released autonomy evaluation resources that include a task suite, software tools, and guidelines for accurately measuring LLM capabilities (METR, 2024). Apollo Research focuses on the evaluation of strategic deception. It finds that under varying levels of stress, GPT-4 engages in illegal behavior such as insider trading and lying about its actions (Scheurer et al., 2023). This finding demonstrates that AI systems may adopt strategies that humans do not approve of in order to help themselves.

Part of