Amazon's voice assistant Alexa ordered items including dollhouses and cookies when triggered by TV news broadcasts discussing the wake word, causing unintended purchases across multiple households.
In January 2017, Amazon's voice-activated Echo devices with Alexa assistant caused multiple incidents of unintended purchases. The initial incident involved a 6-year-old girl in Dallas, Texas who asked her family's Echo Dot 'Can you play dollhouse with me and get me a dollhouse?' which resulted in Alexa ordering a $170 KidKraft Sparkle Mansion dollhouse and four pounds of sugar cookies. When CW6 News in San Diego reported this story, anchor Jim Patton said 'I love the little girl saying Alexa ordered me a dollhouse' on air. This triggered Echo devices in viewers' homes across San Diego to attempt ordering dollhouses as well. The devices responded because voice purchasing was enabled by default and Alexa could not distinguish between different voices or recognize TV audio versus direct commands. Amazon confirmed that while devices may wake up from TV broadcasts, purchases require confirmation with a 'yes' response. The company advised users to manage shopping settings through the Alexa app, including turning off voice purchasing or requiring confirmation codes. Similar incidents occurred with other TV commercials and broadcasts mentioning Alexa commands. The family in the original incident activated parental controls requiring a four-digit code and donated the dollhouse to charity while keeping the cookies.
Domain classification, causal taxonomy, severity scores, and national security assessments were LLM-classified and may contain errors.
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
AI system
Due to a decision or action made by an AI system
Unintentional
Due to an unexpected outcome from pursuing a goal
Post-deployment
Occurring after the AI model has been trained and deployed