Technical vulnerabilities (Robustness - vulnerability to jailbreaking
Vulnerabilities that can be exploited in AI systems, software development toolchains, and hardware, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
"Individuals can manipulate models into performing actions that violate the model’s usage restrictions—a phenomenon known as “jailbreaking.” These manipulations may result in causing the model to perform tasks that the developers have explicitly prohibited (see section 3.2.1.). For instance, users may ask the model to provide information on how to conduct illegal activities— asking for detailed instructions on how to build a bomb or create highly toxic drugs."(p. 62)
Supporting Evidence (1)
"Common forms of malicious attacks231 include: • inputting carefully crafted prompts that are able to navigate around a model’s safeguards,232 • extracting training data (especially sensitive information), • backdooring (negating normal authentication procedures to gain unauthorized access to a system), • data poisoning (intentionally compromising a training dataset to manipulate the operation of a model (see below section 3.1.2.B.3.)), and • exfiltration (the theft or unauthorized removal or movement of data).233"(p. 62)
Part of Technical and operational risks
Other risks from G'sell (2024) (33)
Technical and operational risks
7.3 Lack of capability or robustnessTechnical and operational risks > Technical vulnerabilities (Robustness - unexpected behaviour)
7.3 Lack of capability or robustnessTechnical and operational risks > Technical vulnerabilities (The risk of misalignment)
7.1 AI pursuing its own goals in conflict with human goals or valuesTechnical and operational risks > Factually incorrect content (inaccuracies and fabricated sources)
3.1 False or misleading informationTechnical and operational risks > Opacity (the black box problem)
7.4 Lack of transparency or interpretabilityTechnical and operational risks > Opacity (industry opacity)
6.4 Competitive dynamics