This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Red teaming, capability evaluations, adversarial testing, and performance verification.
Also in Risk & Assurance
Mitigations that are effective in one context – for example, one deployment form or one language – may not be effective in another. As such, developers can assess mitigations in all contexts in which they will be deployed (e.g. API with fine-tuning, API with internet access and tool use, etc.) and assess if mitigations remain effective across natural languages.
Reasoning
Evaluates mitigation effectiveness across deployment contexts and natural languages through comprehensive testing.
Deployment Mitigations
“Articulate how risk mitigations will be identified and implemented to keep risks within defined thresholds, including safety […] mitigations such as modifying system behaviours.”
1 AI SystemDeployment Mitigations > 1. Applying “safety by design” principles
Mitigating deployment risk does not mean waiting until after training to implement measures, but includes measures built into the system design and training process, such as data filtering (CISA, 2013).
1.1.1 Training DataDeployment Mitigations > Building in redundancy
To increase robustness, developers can aim to avoid single points of failure and instead use a “defense-in-depth” strategy, i.e. implementing multiple mitigations addressing the same threat model or failure mode (Alaga et al., 2024).
1 AI SystemDeployment Mitigations > Considering both internal and external deployment:
Different mitigations may be suitable for internal and external deployment due to different underlying risk models. As such, developers may want to separately specify which mitigations they plan to implement for internal use (e.g. monitoring and logging of interactions, limited affordances, staff training).
2.3.1 Deployment ManagementApplying cybersecurity standards
There are many existing cybersecurity standards that developers can apply to the protection of model weights, such as NCSC’s guidelines on secure model development (NCSC, 2024) and DSIT’s draft AI cyber security Code of Practice (DSIT, 2024). Developers can also refer to RAND’s state-of-the-art overview of security measures specifically focused on model weights (Nevo et al., 2024).
3.2.2 Technical StandardsBuilding in redundancy
As with deployment safeguards, developers can aim to implement a “defense-in-depth” strategy.
2.3 Operations & SecurityEmerging Practices in Frontier AI Safety Frameworks
Buhl, Marie Davidsen; Bucknall, Ben; Masterson, Tammy (2025)
As part of the Frontier AI Safety Commitments agreed to at the 2024 AI Seoul Summit, many AI developers agreed to publish a safety framework outlining how they will manage potential severe risks associated with their systems. This paper summarises current thinking from companies, governments, and researchers on how to write an effective safety framework. We outline three core areas of a safety framework - risk identification and assessment, risk mitigation, and governance - and identify emerging practices within each area. As safety frameworks are novel and rapidly developing, we hope that this paper can serve both as an overview of work to date and as a starting point for further discussion and innovation.
Verify and Validate
Testing, evaluating, auditing, and red-teaming the AI system
Developer
Entity that creates, trains, or modifies the AI system
Measure
Quantifying, testing, and monitoring identified AI risks
