This page is still being polished. If you have thoughts, please share them via the feedback form.
Data on this page is preliminary and may change. Please do not share or cite these figures publicly.
Runtime behavior observation, anomaly detection, and activity logging.
Also in Non-Model
Stage: Detection; Stakeholder: Compute Providers; Additional information: Early detection could also be improved by robust real-time monitoring tools that log outputs, decisions and compute usage to detect potential anomalies (Kaur et al. 2023; Greenblatt, Shlegeris et al. 2024). Governments should enhance awareness and information sharing between all stakeholders, including the tracking of compute resources.
Reasoning
Anomaly detection system monitors compute activity for suspicious behavior patterns.
Monitor critical capability levels
2.2.2 Testing & EvaluationIdentify early warning signs and emergent capabilities
2.2.1 Risk AssessmentEstablish standardised benchmarks and reporting
3.2.1 Benchmarks & EvaluationEnhance hardware and supply chain oversight
2.3.3 Monitoring & LoggingLead efforts to establish shared criteria for AI LOC
3.2.2 Technical StandardsCoordinate evaluations and safety testing
2.2.2 Testing & EvaluationStrengthening Emergency Preparedness and Response for AI Loss of Control Incidents
Somani, Elika; Friedman, Anjay; Wu, Henry; Lu, Marianne; Byrd, Christopher; van Soest, Henri; Zakaria, Sana (2025)
As artificial intelligence (AI) systems become increasingly embedded in essential infrastructure and services, the risks associated with unintended failures rise. Developing comprehensive emergency response protocols could help mitigate these significant risks. This report focuses on understanding and addressing AI loss of control (LOC) scenarios where human oversight fails to adequately constrain an autonomous, general-purpose AI.
Other (multiple stages)
Applies across multiple lifecycle stages
Infrastructure Provider
Entity providing compute, platforms, or tooling for AI systems
Measure
Quantifying, testing, and monitoring identified AI risks
