Walmart's AI-powered theft-detection system by Everseen produces false positives that require employee intervention at self-checkout stations, creating health risks during the COVID-19 pandemic by forcing unnecessary close contact between workers and customers.
Walmart deployed an AI-based theft-deterrence system developed by Ireland-based company Everseen across over 2,000 stores starting three years ago. The system uses cameras and computer vision algorithms to monitor customer movements at self-checkout kiosks, detecting when items are bagged but not scanned and alerting hosts to intervene. However, the system generates false positives, flagging legitimate scans as non-scans and prompting workers to step in unnecessarily. During the COVID-19 pandemic, these false alerts have become a health concern as they force employees to break social distancing guidelines and interact closely with customers when intervention isn't actually needed. While CDC recommends maintaining six feet of distance, the system's errors require workers to approach customers and help with scanning, potentially increasing COVID-19 transmission risk. Many grocery workers have fallen ill during the pandemic, with the United Food and Commercial Workers union estimating 65 grocery worker deaths and nearly 10,000 infections from COVID-19. Walmart had to temporarily close a Worcester, Massachusetts location where over 80 employees tested positive for the virus. The company maintains the system isn't problematic based on their data evaluation and notes other reasons also prompt customer assistance.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed