This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Independent audits, third-party reviews, and regulatory compliance verification.
Also in Risk & Assurance
As the application of LLMs in high-stake domains such as healthcare, finance, and law continues to increase, it is imperative to not only assess their accuracy but also scrutinize their safety and reliability (Li et al., 2023e; Liu et al., 2023b). From a societal perspective, the widespread adoption of LLMs across various domains presents potential risks. These risks could arise from a disconnect between LLM developers and users. The former often prioritize technological advancements over practical applications, while the latter may introduce LLMs into their fields without sufficient safety measures or proven success replication. Therefore, Mökander et al. (2023) have proposed that model safety should be audited by third-party entities to rapidly identify risks within LLM systems and issue safety alerts. The auditing process for model safety comprises three steps: • Governance Audit: This involves evaluating the design and dissemination of LLMs to ensure their compliance with relevant legal and ethical standards. • Model Review: This step entails a thorough examination of the LLMs themselves, including aspects such as performance, safety, and fairness. • Application Review: This involves assessing applications based on LLMs to ensure their reliability and safety in practical use.
Reasoning
Interpretability techniques applied during model safety auditing to evaluate and test system behavior.
Value Misalignment
99.9 OtherValue Misalignment > Mitigating social bias
1 AI SystemValue Misalignment > Privacy protection
1 AI SystemValue Misalignment > Methods for mitigating toxicity
1 AI SystemValue Misalignment > Methods for mitigating LLM amorality
1 AI SystemRobustness to attack
1 AI SystemLarge Language Model Safety: A Holistic Survey
Shi, Dan; Shen, Tianhao; Huang, Yufei; Li, Zhigen; Leng, Yongqi; Jin, Renren; Liu, Chuang; Wu, Xinwei; Guo, Zishan; Yu, Linhao; Shi, Ling; Jiang, Bojian; Xiong, Deyi (2024)
The rapid development and deployment of large language models (LLMs) have introduced a new frontier in artificial intelligence, marked by unprecedented capabilities in natural language understanding and generation. However, the increasing integration of these models into critical applications raises substantial safety concerns, necessitating a thorough examination of their potential risks and associated mitigation strategies. This survey provides a comprehensive overview of the current landscape of LLM safety, covering four major categories: value misalignment, robustness to adversarial attacks, misuse, and autonomous AI risks. In addition to the comprehensive review of the mitigation methodologies and evaluation resources on these four aspects, we further explore four topics related to LLM safety: the safety implications of LLM agents, the role of interpretability in enhancing LLM safety, the technology roadmaps proposed and abided by a list of AI companies and institutes for LLM safety, and AI governance aimed at LLM safety with discussions on international cooperation, policy proposals, and prospective regulatory directions. Our findings underscore the necessity for a proactive, multifaceted approach to LLM safety, emphasizing the integration of technical solutions, ethical considerations, and robust governance frameworks. This survey is intended to serve as a foundational resource for academy researchers, industry practitioners, and policymakers, offering insights into the challenges and opportunities associated with the safe integration of LLMs into society. Ultimately, it seeks to contribute to the safe and beneficial development of LLMs, aligning with the overarching goal of harnessing AI for societal advancement and well-being. A curated list of related papers has been publicly available at https://github.com/tjunlp-lab/Awesome-LLM-Safety-Papers.
Verify and Validate
Testing, evaluating, auditing, and red-teaming the AI system
Governance Actor
Regulator, standards body, or oversight entity shaping AI policy
Measure
Quantifying, testing, and monitoring identified AI risks
