This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Version control, prototyping, secure development practices, and engineering processes.
Also in Engineering & Development
Stage: Containment and Mitigation; Stakeholder: National Government: AISI; Additional information: Government stakeholders should seek to strengthen AI security to protect model weights and algorithmic techniques. Governments could require or incentivise AI developers that exceed specified capability thresholds to implement stricter security protections -- of both model weights and algorithmic insights -- to prevent the theft of dangerous capabilities by malicious actors and the diffusion of models to unmonitored environments. Security improvements could include measures such as hardened bandwidth limitations, automated network monitoring and encryption. Routine hardware supply chain and data centre inspections would also help to detect unauthorised access. Beyond external threats, organisations must also consider security risks from the AI models themselves and thus employ regular memory wiping, adversarial testing and monitoring.
Reasoning
Developing security measures for deployments establishes secure development practices within organizational workflows.
Monitor critical capability levels
2.2.2 Testing & EvaluationIdentify early warning signs and emergent capabilities
2.2.1 Risk AssessmentEstablish standardised benchmarks and reporting
3.2.1 Benchmarks & EvaluationImplement compute monitoring and anomaly detection
1.2.3 Monitoring & DetectionEnhance hardware and supply chain oversight
2.3.3 Monitoring & LoggingLead efforts to establish shared criteria for AI LOC
3.2.2 Technical StandardsStrengthening Emergency Preparedness and Response for AI Loss of Control Incidents
Somani, Elika; Friedman, Anjay; Wu, Henry; Lu, Marianne; Byrd, Christopher; van Soest, Henri; Zakaria, Sana (2025)
As artificial intelligence (AI) systems become increasingly embedded in essential infrastructure and services, the risks associated with unintended failures rise. Developing comprehensive emergency response protocols could help mitigate these significant risks. This report focuses on understanding and addressing AI loss of control (LOC) scenarios where human oversight fails to adequately constrain an autonomous, general-purpose AI.
Build and Use Model
Training, fine-tuning, and integrating the AI model
Governance Actor
Regulator, standards body, or oversight entity shaping AI policy
Manage
Prioritising, responding to, and mitigating AI risks