An AI gun detection system at Kenwood High School in Baltimore County mistakenly identified a student's bag of Doritos as a firearm, leading to armed police handcuffing and searching the student.
On Monday evening at Kenwood High School in Baltimore County, Maryland, an AI-powered gun detection system operated by Omnilert flagged a student's empty bag of Doritos chips as a possible firearm. The system, which has been used in Baltimore County public schools since 2023, analyzes video from existing security cameras to detect potential weapons. Student Taki Allen was waiting with friends for a ride home after football practice when approximately eight police cars arrived at the school. Armed officers approached Allen with guns drawn, forced him to his knees, handcuffed and searched him before determining he was unarmed. The AI system had detected Allen holding the chip bag with 'two hands and one finger out' which resembled a gun to the algorithm. While the school's Department of School Safety and Security reviewed and canceled the alert after confirming there was no weapon, the principal had already contacted the school resource officer who called local police for support. The principal was not immediately aware the alert had been canceled. Baltimore County officials and councilmembers are now calling for a review of the AI gun detection system and the response procedures that led to the traumatic incident.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed