This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Modifications to training data composition, quality, and filtering that affect what the model learns.
Also in Model
Careful data curation prior to all development stages (including fine-tuning) to filter out high-risk content and ensure the training data is sufficiently high-quality.
Multiple experts agreed that this measure could be effective for reducing bias and discrimination risks, with some noting it as a fundamental or critical step. Several experts also saw potential benefits for mitigating chemical, biological, radiological and nuclear (CBRN) risks, particularly by filtering out specialised data. However, opinions were mixed on its effectiveness for other risk areas. Some experts expressed scepticism about its impact on democratic process risks and critical infrastructure disruption. Several noted the difficulty in defining and identifying "high-risk" content in these domains. Several experts highlighted implementation challenges, including the vast amount of data involved, potential reduction in model capabilities, and the difficulty of thorough curation without significantly limiting the training set. Several mentioned that companies likely already attempt some level of data curation for performance reasons. Some experts cautioned that this approach alone is insufficient, as models may still be able to recreate dangerous knowledge from "safe" data. One expert noted that careful data curation might be more effective for near-term models (in the next two years) than for more advanced future systems. Several experts emphasised the importance of balancing risk mitigation with maintaining useful capabilities, noting the dual-use nature of much scientific and technical information. One expert argued against the concept of "high-quality" data, suggesting all data could be useful depending on the application.
Reasoning
Filters harmful content and low-quality data from training corpus before model learning begins.
Pre-deployment risk assessments
Comprehensive risk assessments before deployment that would assess reasonably foreseeable misuse and include dangerous capability evaluations that incorporate post-training enhancements and collaborations with domain experts. Risk assessments would inform deployment decisions.
2.2.1 Risk AssessmentThird party pre-deployment model audits
External pre-deployment assessment to provide a judgment on the safety of a model. Auditors, which could be governments or independent third parties, would receive access to a fine-tuning API for testing, or further appropriate technical means.
2.2.3 Auditing & ComplianceExternal assessment of testing procedure
Bringing in external AI evaluation firms before deployment to assess and red-team the company's execution of dangerous capabilities evaluations.
2.2.2 Testing & EvaluationVetted researcher access
Giving good faith, public interest evaluation researchers access to black-box research APIs that provide technical and legal safe harbours to limit barriers imposed by usage policy enforcement, logging, and stringent terms of service.
2.3.1 Deployment ManagementAdvanced model access for vetted external researchers
Examples of advanced access rights could include any of the following: increased control over sampling, access to fine-tuning functionality, the ability to inspect and modify model internals, access to training data, or additional features like stable model versions.
2.2.2 Testing & EvaluationHarmlessness training
State-of-the-art reinforcement learning and fine-tuning techniques, such as Reinforcement Learning from Human Feedback (RLHF) or Direct Preference Optimization (DPO), to ensure models do not engage in unsafe behavior.
1.1.2 Learning ObjectivesEffective Mitigations for Systemic Risks from General-Purpose AI
Uuk, Risto; Brouwer, Annemieke; Schreier, Tim; Dreksler, Noemi; Pulignano, Valeria; Bommasani, Rishi (2024)
The systemic risks posed by general-purpose AI models are a growing concern, yet the effectiveness of mitigations remains underexplored. Previous research has proposed frameworks for risk mitigation, but has left gaps in our understanding of the perceived effectiveness of measures for mitigating systemic risks. Our study addresses this gap by evaluating how experts perceive different mitigations that aim to reduce the systemic risks of general-purpose AI models. We surveyed 76 experts whose expertise spans AI safety; critical infrastructure; democratic processes; chemical, biological, radiological, and nuclear risks (CBRN); and discrimination and bias. Among 27 mitigations identified through a literature review, we find that a broad range of risk mitigation measures are perceived as effective in reducing various systemic risks and technically feasible by domain experts. In particular, three mitigation measures stand out: safety incident reports and security information sharing, third-party pre-deployment model audits, and pre-deployment risk assessments. These measures show both the highest expert agreement ratings (>60\%) across all four risk areas and are most frequently selected in experts' preferred combinations of measures (>40\%). The surveyed experts highlighted that external scrutiny, proactive evaluation and transparency are key principles for effective mitigation of systemic risks. We provide policy recommendations for implementing the most promising measures, incorporating the qualitative contributions from experts. These insights should inform regulatory frameworks and industry practices for mitigating the systemic risks associated with general-purpose AI.
Collect and Process Data
Gathering, curating, labelling, and preprocessing training data
Developer
Entity that creates, trains, or modifies the AI system
Manage
Prioritising, responding to, and mitigating AI risks
