This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Technical mechanisms and engineering interventions that directly modify how an AI system processes inputs, generates outputs, or operates, including changes to models, training procedures, runtime behaviors, and supporting hardware.
“Articulate how risk mitigations will be identified and implemented to keep risks within defined thresholds, including safety […] mitigations such as modifying system behaviours.”
Reasoning
Risk assessment process identifies harms and plans mitigations to maintain acceptable thresholds.
1. Applying “safety by design” principles
Mitigating deployment risk does not mean waiting until after training to implement measures, but includes measures built into the system design and training process, such as data filtering (CISA, 2013).
1.1.1 Training DataBuilding in redundancy
To increase robustness, developers can aim to avoid single points of failure and instead use a “defense-in-depth” strategy, i.e. implementing multiple mitigations addressing the same threat model or failure mode (Alaga et al., 2024).
1 AI SystemConsidering both internal and external deployment:
Different mitigations may be suitable for internal and external deployment due to different underlying risk models. As such, developers may want to separately specify which mitigations they plan to implement for internal use (e.g. monitoring and logging of interactions, limited affordances, staff training).
2.3.1 Deployment ManagementApplying cybersecurity standards
There are many existing cybersecurity standards that developers can apply to the protection of model weights, such as NCSC’s guidelines on secure model development (NCSC, 2024) and DSIT’s draft AI cyber security Code of Practice (DSIT, 2024). Developers can also refer to RAND’s state-of-the-art overview of security measures specifically focused on model weights (Nevo et al., 2024).
3.2.2 Technical StandardsBuilding in redundancy
As with deployment safeguards, developers can aim to implement a “defense-in-depth” strategy.
2.3 Operations & SecurityMitigation evaluation
“Risk assessments should consider […] the efficacy of implemented mitigations.” (I)
2.2.2 Testing & EvaluationMitigation evaluation > Setting specific targets
Being precise about what standards mitigations aim to meet makes testing easier and reduces subjectivity. These targets can be grounded risk modelling (1.2) or based on risk thresholds (1.3).
2.2.4 Assurance DocumentationMitigation evaluation > Evaluating robustly
As with capability elicitation, it can be helpful to consider what level of effort is necessary to provide robust evidence of mitigation effectiveness. For example, the developer can use red-teams that are as capable and well-resourced as relevant threat actors. Developers can also aim to use multiple types of evaluations, where feasible, to increase robustness.
2.2.2 Testing & EvaluationMitigation evaluation > Evaluating comprehensively
Mitigations that are effective in one context – for example, one deployment form or one language – may not be effective in another. As such, developers can assess mitigations in all contexts in which they will be deployed (e.g. API with fine-tuning, API with internet access and tool use, etc.) and assess if mitigations remain effective across natural languages.
2.2.2 Testing & EvaluationEmerging Practices in Frontier AI Safety Frameworks
Buhl, Marie Davidsen; Bucknall, Ben; Masterson, Tammy (2025)
As part of the Frontier AI Safety Commitments agreed to at the 2024 AI Seoul Summit, many AI developers agreed to publish a safety framework outlining how they will manage potential severe risks associated with their systems. This paper summarises current thinking from companies, governments, and researchers on how to write an effective safety framework. We outline three core areas of a safety framework - risk identification and assessment, risk mitigation, and governance - and identify emerging practices within each area. As safety frameworks are novel and rapidly developing, we hope that this paper can serve both as an overview of work to date and as a starting point for further discussion and innovation.
Deploy
Releasing the AI system into a production environment
Deployer
Entity that integrates and deploys the AI system for end users
Manage
Prioritising, responding to, and mitigating AI risks
Primary
7 AI System Safety, Failures & Limitations