Spain's VioGén AI system for assessing domestic violence risk systematically underestimated danger levels, leading to inadequate protection for women who were subsequently murdered by their abusers.
VioGén is an algorithmic risk assessment system deployed by Spain's Ministry of Interior in 2007 to evaluate the likelihood of repeat domestic violence incidents. The system uses a questionnaire with 39 items (in version 4.0) that police officers complete with victims to generate risk scores ranging from 'no risk appreciated' to 'extreme risk.' The algorithm determines what level of police protection women receive after reporting abuse. Multiple audit reports and investigations found severe flaws in the system. In 2014, 14 out of 15 women who were killed after reporting their abusers had been classified as 'low risk' or 'no risk appreciated' by VioGén. The system has been criticized for lack of transparency, minimal human oversight (officers only modify the AI's risk assessment in 5% of cases), systematic underestimation of psychological violence, and adapting risk classifications based on available police resources rather than actual danger levels. An external audit by Eticas Consulting found that in 2021, only 1 out of 7 women who sought police protection actually received it, with approximately 45% of cases receiving 'no risk appreciated' classifications. The system processes over 600,000 cases and has been updated multiple times, most recently in 2019 with version 5.0 that includes specific assessments for lethal violence risk.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed