This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Structured analysis to identify, characterize, and prioritize potential harms and risks.
Also in Risk & Assurance
Stage: Detection; Stakeholder: AI Developers; Additional information: Agreement on early warning signs that may signal a LOC incident would help determine proportional responses to risks (Popoola et al. 2013). Developers and government stakeholders should consider adopting practices from cybersecurity and biosecurity domains by integrating confidence scoring systems and continuous, overlapping detection mechanisms (CISA 2025a; Yousef et al. 2024; Thompson et al. 2019).
Reasoning
Identifying emergent capabilities and warning signs characterizes potential risks before deployment—a structured pre-deployment risk analysis activity.
Monitor critical capability levels
2.2.2 Testing & EvaluationEstablish standardised benchmarks and reporting
3.2.1 Benchmarks & EvaluationImplement compute monitoring and anomaly detection
1.2.3 Monitoring & DetectionEnhance hardware and supply chain oversight
2.3.3 Monitoring & LoggingLead efforts to establish shared criteria for AI LOC
3.2.2 Technical StandardsCoordinate evaluations and safety testing
2.2.2 Testing & EvaluationStrengthening Emergency Preparedness and Response for AI Loss of Control Incidents
Somani, Elika; Friedman, Anjay; Wu, Henry; Lu, Marianne; Byrd, Christopher; van Soest, Henri; Zakaria, Sana (2025)
As artificial intelligence (AI) systems become increasingly embedded in essential infrastructure and services, the risks associated with unintended failures rise. Developing comprehensive emergency response protocols could help mitigate these significant risks. This report focuses on understanding and addressing AI loss of control (LOC) scenarios where human oversight fails to adequately constrain an autonomous, general-purpose AI.
Build and Use Model
Training, fine-tuning, and integrating the AI model
Developer
Entity that creates, trains, or modifies the AI system
Measure
Quantifying, testing, and monitoring identified AI risks
